1857803914604618029
This commit is contained in:
ed committed
1 parent
ab7639672c
commit
a6092de000
6 files changed
+303
No files matched your search
+47
@@ -0,0 +1,47 @@
|
||||
---
|
||||
title: "@NOTimothyLottes So each line is 63 spaces followed by a \\n, and those 63 spaces get replaced by printable characters dynamically? I have long dreamed of a text format with a fixed 128 byte line width, and cache line size is gradually switching over from 64 to 128"
|
||||
type: archive
|
||||
source: twitter
|
||||
source_url: "https://x.com/bmcnett/status/1857815519166112244"
|
||||
author: "bmcnett"
|
||||
handle: bmcnett
|
||||
post_id: "1857815519166112244"
|
||||
date: 2024-11-16
|
||||
archived: 2026-08-25
|
||||
draft: false
|
||||
tags:
|
||||
- archive
|
||||
- twitter
|
||||
- NOTimothyLottes
|
||||
description: "@NOTimothyLottes So each line is 63 spaces followed by a \\n, and those 63 spaces get replaced by printable characters dynamically? I have long dreamed of a text format with a fixed 128 byte line width, and cache line size is gradually switching over from 64 to 128"
|
||||
in_reply_to: ""
|
||||
parent_post_id: "1857813028349128825"
|
||||
---
|
||||
|
||||
## Source
|
||||
|
||||
- URL: https://x.com/bmcnett/status/1857815519166112244
|
||||
- Author: bmcnett (@bmcnett)
|
||||
- Posted: 2024-11-16 15:58:30
|
||||
|
||||
## Branch
|
||||
|
||||
**1/**
|
||||
|
||||
@NOTimothyLottes So each line is 63 spaces followed by a \n, and those 63 spaces get replaced by printable characters dynamically? I have long dreamed of a text format with a fixed 128 byte line width, and cache line size is gradually switching over from 64 to 128
|
||||
|
||||
**2/**
|
||||
|
||||
@bmcnett Yes fixed character[63]='\n', string fills until then or spaces out. I did 128-char once, but still sometimes use small VGA resolutions. Also written a source editor a few times with this fixed line size, makes for a very simple implementation ...
|
||||
|
||||
**3/**
|
||||
|
||||
@NOTimothyLottes Variable line width ASCII text files made a lot of sense when storage was 100KB or 100MB or 100GB, but now that it's in the terabytes, why even bother
|
||||
|
||||
**4/**
|
||||
|
||||
@bmcnett That line of thinking is universal solution to most self-inflicted problems of the modern era.
|
||||
|
||||
## Related
|
||||
|
||||
- Spine: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging]]
|
||||
+60
@@ -0,0 +1,60 @@
|
||||
---
|
||||
title: "@NOTimothyLottes Why the heck do you care about cache lines, if you write to a mmaped file? Of course there's overhead: TLBs aren't free."
|
||||
type: archive
|
||||
source: twitter
|
||||
source_url: "https://x.com/datenwolf/status/1857820149039947846"
|
||||
author: "datenwolf – here to witness Τwitter's death"
|
||||
handle: datenwolf
|
||||
post_id: "1857820149039947846"
|
||||
date: 2024-11-16
|
||||
archived: 2026-08-25
|
||||
draft: false
|
||||
tags:
|
||||
- archive
|
||||
- twitter
|
||||
- NOTimothyLottes
|
||||
description: "@NOTimothyLottes Why the heck do you care about cache lines, if you write to a mmaped file? Of course there's overhead: TLBs aren't free."
|
||||
in_reply_to: ""
|
||||
parent_post_id: "1857805438151983120"
|
||||
---
|
||||
|
||||
## Source
|
||||
|
||||
- URL: https://x.com/datenwolf/status/1857820149039947846
|
||||
- Author: datenwolf – here to witness Τwitter's death (@datenwolf)
|
||||
- Posted: 2024-11-16 16:16:54
|
||||
|
||||
## Branch
|
||||
|
||||
**1/**
|
||||
|
||||
@NOTimothyLottes Why the heck do you care about cache lines, if you write to a mmaped file? Of course there's overhead: TLBs aren't free. And TLBs matter… at lot in fact. When you map the file, initially the kernel marks some address space, but doesn't populate it with pages. 1/
|
||||
|
||||
**2/**
|
||||
|
||||
@NOTimothyLottes On the first write to a non-faulted page, the kernel has to find a free page in memory, associate it in the page table, all the while keeping an eye on all threads process, since changes to a process virtual address space must appear atomic. 2/
|
||||
|
||||
**3/**
|
||||
|
||||
@NOTimothyLottes Memory maps aren't free. Simple writes to a file will easily outperform and cause less overhead, than naively implemented mmaped I/O. 3/
|
||||
|
||||
**4/**
|
||||
|
||||
@datenwolf BTW, I pre-fault my mapped files. Just like I pre-fault the binary and data segments too. But sure the pre-fault could go stale. And yes TLBs are a system perf issue, but atlas you need admin priv on Windows to play 2 MiB pages for data, so it's off the table
|
||||
|
||||
**5/**
|
||||
|
||||
@datenwolf Why cacheline aligned and sized on the CPU? In theory store the entire line in a short enough timeframe and your Read-Modify-Write becomes just a Write! CPU doesn't have byte-write-mask (I think). If SW+HW is good. And threads keeping to isolated lines means no false sharing. Etc
|
||||
|
||||
**6/**
|
||||
|
||||
@NOTimothyLottes (That cacheline remark was meant rhetorically; between cache misses and page faults, a page fault will largely overshadow a cache miss in performance overhead.
|
||||
But yes, if you prefault, then that problem is mitigated.)
|
||||
|
||||
**7/**
|
||||
|
||||
@datenwolf 'Smart' people keep trying to impose (aka force) their will on others through poor system APIs, like not being able to pre-pin pages. Good look buddy, I'll just run a background task that continuously touch walks the pages I want pinned anyway.
|
||||
|
||||
## Related
|
||||
|
||||
- Spine: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging]]
|
||||
+55
@@ -0,0 +1,55 @@
|
||||
---
|
||||
title: "@NOTimothyLottes This is super cool, but why map a file and not just stdout?"
|
||||
type: archive
|
||||
source: twitter
|
||||
source_url: "https://x.com/static_assert_0/status/1857867668289663108"
|
||||
author: "static_assert(false)"
|
||||
handle: static_assert_0
|
||||
post_id: "1857867668289663108"
|
||||
date: 2024-11-16
|
||||
archived: 2026-08-25
|
||||
draft: false
|
||||
tags:
|
||||
- archive
|
||||
- twitter
|
||||
- NOTimothyLottes
|
||||
description: "@NOTimothyLottes This is super cool, but why map a file and not just stdout?"
|
||||
in_reply_to: ""
|
||||
parent_post_id: "1857803914604618029"
|
||||
---
|
||||
|
||||
## Source
|
||||
|
||||
- URL: https://x.com/static_assert_0/status/1857867668289663108
|
||||
- Author: static_assert(false) (@static_assert_0)
|
||||
- Posted: 2024-11-16 19:25:44
|
||||
|
||||
## Branch
|
||||
|
||||
**1/**
|
||||
|
||||
@NOTimothyLottes This is super cool, but why map a file and not just stdout?
|
||||
|
||||
**2/**
|
||||
|
||||
@static_assert_0 @NOTimothyLottes The writes can be done concurrently from many threads?
|
||||
|
||||
**3/**
|
||||
|
||||
@vodangkhoa873 @static_assert_0 Yes, one 'atomicAdd(atomAdr,1)&65535' to get the line number (window of 65536 max lines) then write the line individually. Only contention is on that atomicAdd CPU instruction which is as minimal as possible.
|
||||
|
||||
I do the atomic add instruction using simple inline ASM (below)
|
||||
|
||||
|
||||
|
||||
**4/**
|
||||
|
||||
@NOTimothyLottes @vodangkhoa873 I love this, but are you not able to get that codegen with intrinsics or just normal C?
|
||||
|
||||
**5/**
|
||||
|
||||
@static_assert_0 @vodangkhoa873 I've been hand rolling intrinsics CPU-side in GCC with inline asm since the dawn of support for that. So haven't kept up with the high-level-lang mess fire since.
|
||||
|
||||
## Related
|
||||
|
||||
- Spine: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging]]
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
---
|
||||
title: "@NOTimothyLottes Curious why you memory map the 32 bit counter to the file."
|
||||
type: archive
|
||||
source: twitter
|
||||
source_url: "https://x.com/HerrUppoHoppa/status/1858039804551663740"
|
||||
author: "Pontus"
|
||||
handle: HerrUppoHoppa
|
||||
post_id: "1858039804551663740"
|
||||
date: 2024-11-17
|
||||
archived: 2026-08-25
|
||||
draft: false
|
||||
tags:
|
||||
- archive
|
||||
- twitter
|
||||
- NOTimothyLottes
|
||||
description: "@NOTimothyLottes Curious why you memory map the 32 bit counter to the file."
|
||||
in_reply_to: ""
|
||||
parent_post_id: "1857804669994565771"
|
||||
---
|
||||
|
||||
## Source
|
||||
|
||||
- URL: https://x.com/HerrUppoHoppa/status/1858039804551663740
|
||||
- Author: Pontus (@HerrUppoHoppa)
|
||||
- Posted: 2024-11-17 06:49:44
|
||||
|
||||
## Branch
|
||||
|
||||
**1/**
|
||||
|
||||
@NOTimothyLottes Curious why you memory map the 32 bit counter to the file. Also why does it get half the file as opposed to just the end of the mapped region?
|
||||
|
||||
**2/**
|
||||
|
||||
@HerrUppoHoppa It's pow2 message lines so fixing wrap is AND(cnt,lines-1). Counter after that and needs to take a page minimum. So yes I could make the size slightly smaller, but just lazy. Counter in file because then multiple runs accumulate in the file for comparison.
|
||||
|
||||
**3/**
|
||||
|
||||
@NOTimothyLottes Ah I see, interesting solution!
|
||||
|
||||
## Related
|
||||
|
||||
- Spine: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging]]
|
||||
+97
@@ -0,0 +1,97 @@
|
||||
---
|
||||
title: "[0] What a horrible term 'printf-debugging'"
|
||||
type: archive
|
||||
source: twitter
|
||||
source_url: "https://x.com/NOTimothyLottes/status/1857803914604618029"
|
||||
author: "NOTimothyLottes"
|
||||
handle: NOTimothyLottes
|
||||
post_id: "1857803914604618029"
|
||||
date: 2024-11-16
|
||||
archived: 2026-08-25
|
||||
draft: false
|
||||
tags:
|
||||
- archive
|
||||
- twitter
|
||||
- NOTimothyLottes
|
||||
description: "[0] What a horrible term 'printf-debugging'"
|
||||
in_reply_to: ""
|
||||
---
|
||||
|
||||
## Source
|
||||
|
||||
- URL: https://x.com/NOTimothyLottes/status/1857803914604618029
|
||||
- Author: NOTimothyLottes (@NOTimothyLottes)
|
||||
- Posted: 2024-11-16 15:12:24
|
||||
|
||||
## Thread
|
||||
|
||||
**1/**
|
||||
|
||||
[0] What a horrible term 'printf-debugging'
|
||||
If you are debugger-free might as well at least be libc-free
|
||||
Here is how I work CPU C code without any debugger in my engines, and how it works better than a debugger for me ...
|
||||
|
||||
Branches: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging/2024-11-16-static_assert_0-this-is-super-cool-but-why-map-a-file-and-not]]
|
||||
|
||||
**2/**
|
||||
|
||||
[1] First I map a '.log' file, so there is no other file IO
|
||||
(a.) First half of this file (log2 sized file) contains lines of a fixed 64-character size (with the \n at the end)
|
||||
(b.) Second half contains just one 32-bit atomic counter of where to write a line
|
||||
|
||||
Branches: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging/2024-11-17-HerrUppoHoppa-curious-why-you-memory-map-the-32-bit-counter-to]]
|
||||
|
||||
**3/**
|
||||
|
||||
[2] So writing a line to this log is 100% overhead-free and lock-free, one just increments the atomic counter, and dumps the comment line in the memory mapped file. Note the 64-byte line size matches a cacheline size, so no contention on the write of the line
|
||||
|
||||
Branches: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging/2024-11-16-datenwolf-why-the-heck-do-you-care-about-cache-lines-if-you]]
|
||||
|
||||
**4/**
|
||||
|
||||
[3] It's ideal for a multi-thread environment, you get correct temporal ordering captured in the log file without any real effect on the scheduling of threads.
|
||||
|
||||
**5/**
|
||||
|
||||
[4] Lines are always in a fixed format
|
||||
r|sc.milmic|line_|hex_____|0000000000-|string...
|
||||
r ... reload count
|
||||
sc.milmic ... time since launch (wraps)
|
||||
line ... source line number (single file program)
|
||||
hex ... hex print of number
|
||||
000 ... decimal print of the number
|
||||
string ... user msg
|
||||
|
||||
**6/**
|
||||
|
||||
[5] I write a line with a simple function call
|
||||
Tlk(__LINE__,n,"msg");
|
||||
Where 'n' is the number, "msg" is a string
|
||||
Don't need printf, this is way better
|
||||
|
||||
**7/**
|
||||
|
||||
[6] Here is an example with a very simple program, just a few first lines. It's nice to automatically see exact timing. Timing since start
|
||||
0.005478 sec ... mapping of the cartridge file
|
||||
0.006850 sec ... page faulting the full cart file
|
||||
0.008230 sec ... window creation done
|
||||
etc
|
||||
|
||||
|
||||
|
||||
**8/**
|
||||
|
||||
[7] Another simple example of a fully pipelined startup, can instantly know where the primary cost is,
|
||||
"Instance begin" and "Instance end" wrap setup of the Vulkan instance, a process which took 2.4 seconds. The 4 PSOs hit in the shader cache at 3k microseconds each
|
||||
|
||||
|
||||
|
||||
**9/**
|
||||
|
||||
[8] The log file keeps multiple runs, so it's easy to compare. Notepad2 it's F5 to reload the log file.
|
||||
|
||||
Branches: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging/2024-11-16-bmcnett-so-each-line-is-63-spaces-followed-by-a-n-and]]
|
||||
|
||||
**10/**
|
||||
|
||||
[9] The point: keep it minimal, keep it simple, and keep the number of required things that take human time to the smallest possible. If you don't fall off the 'compile-time' curve, and re-compile/re-start/re-load is 'key-press-time' this works amazingly well (interactive)
|
||||
@@ -28,6 +28,7 @@ tags:
|
||||
- [[archive/threads/NOTimothyLottes/2023-12-19-multi-session-is-nice-since-the-binary-auto]]
|
||||
- [[archive/threads/NOTimothyLottes/2024-04-06-on-https-mastodon-gamedev-place-notimothylottes]]
|
||||
- [[archive/threads/NOTimothyLottes/2024-05-04-it-was-only-later-when-systems-forced-c-abi]]
|
||||
- [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging]]
|
||||
- [[archive/threads/NOTimothyLottes/2024-11-16-that-line-of-thinking-is-universal-solution-to]]
|
||||
- [[archive/threads/NOTimothyLottes/2024-11-19-after-vk-device-is-open-i-signal-a-background]]
|
||||
- [[archive/threads/NOTimothyLottes/2024-11-21-32-oh-damn-32-should-probably-stop-now-until-next]]
|
||||
|
||||
Reference in new issue
Block a user