ed/SectrPrototype
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

preparing for codebase rewrite

Browse Source
This commit is contained in:
Edward R. Gonzalez
2025-09-14 16:05:30 -04:00
parent 6c6e4ad75e
commit 8125f1680c
19 changed files with 17 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

188
code/vefontcache/docs/Readme.md Normal file
View File

@@ -0,0 +1,188 @@
# Interface
## Lifetime
### startup
Initializes a provided context.
There are a large amount of parameters to tune the library instance to the user's preference. By default, keep in mind the library defaults to utilize stb_truetype as the font parser and harfbuzz for the shaper.
Much of the data structures within the context struct are not fixed-capacity allocations so make sure that the backing allocator can handle it.
### hot_reload
The library supports being used in a dynamically loaded module. If its hot-reloaded simply make sure to call this procedure with a reference to the backing allocator provided during startup as all dynamic containers tend to lose a proper reference to the allocator's procedure.
Call `clear_atlas_region_caches` & `clear_shape_cache` to reset the library's shape and glyph cache state to force a re-render.
### shutdown
Release resources from the context.
### clear_atlas_region_caches
Clears the LRU caches of regions A-D of the Atlas & sets their next_idx to 0. Effectively will force a re-cache of all previously rendered glyphs. Shape configuration for the glyph will remain unchanged unless clear_shape_cache is also called.
### clear_shape_cache
Clears the LRU cache of the shaper along with clearing all existing storage entries. Effectively will force a re-cache of previously cached text shapes (Does not recache their rendered glyphs).
### load_font
Will load an instance of a font. The user needs to load the file's bytes themselves, the font entry (Entry :: struct) will by tracked by the library. The user will be given a font_id which is a direct index for the entry in the tracked array.
### unload_font
Will free an entry, (parser and shaper resources also freed)
## Shaping
Ideally the user should track the shapes themselves in a time-scale beyond the per-frame draw call. This avoids having to do caching/lookups of the shope.
### shape_text
Will shape the text using the `shaper_proc` arugment (user overloadable). Shape will be cached by the library.
### shape_text_uncached
Will shape the text using the `shaper_proc` arugment (user overloadable).
Shape will NOT be cached by the library. Use this if you want to roll your own solution for tracking shapes.
## Draw list generation
### draw_text procedures
There a total of six procedures, 3 for shapes, 3 for text:
* `draw_shape_normalized_space`
* `draw_shape_view_space`
* `draw_shape`
* `draw_text_normalized_space`
* `draw_text_view_space`
* `draw_text`
The normalized space procedures are the `baseline` interface draw procedures. They expec the position, and scale provided to operate with an unsigned normalized space where the bottom left is 0.0, 0.0 and the top right is 1.0, 1.0.
The view space will normalize the position and scale for the user based on the provided view and zoom. The coordinate system is still unsigned just scaled to the view's size.
The non-suffix named procedures use the scope stack to derive the position and scale the user provides a relative position and scale for the text that will be adjusted to the scope's view, position, scale, & zoom.
See the comment above each of the procedures for diagrams.
### get_draw_list
Get the enqueded draw_list (vertices, indices, and draw call arrays) in its entirety.
By default, if get_draw_list is called, it will first call `optimize_draw_list` to optimize the draw list's calls for the user. If this is undesired, make sure to pass `optimize_before_returning = false` in the arguments.
### get_draw_list_layer
Get the enqueued draw_list for the current "layer".
A layer is considered the slice of the `Draw_List`'s content from the last call to `flush_draw_list_layer` onward.
By default, if `get_draw_list_layer` is called, it will first call `optimize_draw_list` for the user to optimize the slice (exlusively) of the draw list's draw calls. If this is undesired, make sure to pass `optimize_before_returning = false` in the arguments.
The draw layer offsets are cleared with `flush_draw_list`
### flush_draw_list
Will clear the draw list and draw layer offsets.
### flush_draw_list_layer
Will update the draw list layer with the latest offset based on the current lenght of the draw list vertices, indices, and calls arrays.
## Metrics
### measure_shape_size
This provide's the shape size scaled down by the ctx.px_scale to get intended usage size. Size is equivalent to `measure_text_size`.
### measure_text_size
Provides a Vec2 the width and height occupied by the provided text string. The y is measured to be the the largest glyph box bounds height of the text. The width is derived from the `end_cursor_pos` field from a `Shaped_Text` entry.
### get_font_vertical_metrics
A wrapper for `parser_get_font_vertical_metrics`. Will provide the ascent, descent, and line_gap for a font entry.
## Miscellaneous
Stuff used by the draw list generation interface or just getters and setters.
### get_cursor_pos
Will provide the current cursor_pos for the resulting text drawn.
### get_normalized_position_scale
Will normalize the value of the position and scale based on the provided view.
Position will also be snapped to the nearest pixel via ceil.
Does nothing if view is 1 or 0
This is used by draw via view relative space procedures to normalize it to the intended space for the render pass.
### resolve_draw_px_size
Used to constrain the px_size used in `resolve_zoom_size_scale`.
The view relative space and scoping stack-based procedures support zoom. When utilizing zoom their is a nasty jitter that will occur if the user smoothly goes across different font sizes because the spacing can drastically change between even and odd font-sizes. This is applied to enforce the font sticks to a specific interval.
The library uses the context's zoom_px_interval as the reference interval in the draw procedures. It can be set with `set_zoom_px_interval` and the default value is 2.
### resolve_zoom_size_scale
Provides a way to get a "zoom" on the font size and scale, similar conceptually to a canvas UX zoom
Does nothing when zoom is 1.0
Uses `resolve_draw_px_size` to constrain which font size is used for the zoom.
### set_alpha_scalar
This is an artifact feature of the current shader, it *may* be removed in the future... Increasing the alpha of the colour draw with above 1.0 increases the edge contrast of the glyph shape.
For the value to be added to the colour, the alph of the text must already be at 1.0 or greater.
### set_px_scalar
This another "super-scalar" applied to rendering glyphs. In each draw procedure the following is computed before passing the values to the shaper and draw list generation passes:
```go
target_px_size := px_size * ctx.px_scalar
target_scale := scale * (1 / ctx.px_scalar)
target_font_scale := parser_scale( entry.parser_info, target_px_size )
```
Essentially, `ctx.px_scalar` is used to upscale the px_size by its value and then downscale the render target scale back the indended size. Doing so provides better shape positioning and futher improves text hinting. The downside is that small text tends to become more jagged (as its really hitting the limits of of how well the shader can blend those edges at that resolution).
This will most likely be preserved with future shader upgrades, however it will most likely not be as necessary as it is right now to achieve crisp text.
### set_zoom_px_interval
Used with by draw procedures with `resolve_draw_px_size` & `resolve_zoom_size_scale`. Provides the interval to use when constraining the px_size to a specific set of values when using zoom scaling.
### set_snap_glyph_shape_position
During the shaping pass, the position of each glyph can be rounded up to the integer to (ussually) allow better hinting.
### set_snap_glyph_render_height
During the draw list generation pass, the position of each glyph when blitting to atlas can have teh quad size rounded up to the integer.
Can yield better hinting but may significantly stretch the glyphs at small scales.
## Scope Stack
These are a set of push & pop pairs of functions that operator ont he context's stack containers. They are used with the draw_shape and draw_text procedures. This mainly for quick scratch usage where the user wants to directly compose a large amount of text without having a UI framework directly handle the text backend.
* font
* font_size
* colour: Linear colour.
* view: Width and height of the 2D area the text will be drawn within.
* position: Uses relative positioning will offset the incoming position by the given amount.
* scale: Uses relative scaling, will scale the procedures incoming scale by the given amount.
* zoom: Affects scaling, will scale the procedure's incoming font size & scale based on an *UX canvas camera's* notion of it.
Procedure types:
* `scope_<stack_option>`: push with a defer pop
* `push_<stack_option>`
* `pop_<stack_option>`

234
code/vefontcache/docs/guide_architecture.md Normal file
View File

@@ -0,0 +1,234 @@
# Guide: Architecture
Overview of the package design and code-path layout.
---
The purpose of this library is to alleviate four key challenges with one encapsulating package:
* Font parsing
* Text codepoint shaping
* Glyph shape triangulation
* Glyph draw-list generation
Shaping text, getting metrics for glyphs, triangulating glyphs, and anti-aliasing their render are expensive operations to perform per frame. Therefore, any compute operations that can be cached, will be.
There are two cache types used:
* Shape cache (`Shaped_Text_Cache.state`)
* Atlas region cache (`Atlas_Region.state`)
The shape cache stores all data for a piece of text that will be utilized in a draw call that is not dependent on a specific position & scale (and is faster to lookup vs compute per draw call).
The atlas region cache tracks what slots have glyphs rendered to the texture atlas. This essentially caches triangulation and super-sampling computations.
All caching uses [LRU.odin](../vefontcache/LRU.odin)
## Code Paths
### Lifetime
The library lifetime is straightforward: you have a startup procedure that should be called during your usual app initialization. From there you may either choose to manually shut it down or let the OS clean it up.
If hot-reload is desired, you just need to call hot_reload with the context's backing allocator to refresh the procedure references. After the DLL has been reloaded, these should be the only aspects that have been scrambled.
Usually when hot-reloading the library for tuning or major changes, you'd also want to clear the caches. Simply call `clear_atlas_region_caches` & `clear_shape_cache` right after.
Ideally, there should be zero dynamic allocation on a per-frame basis as long as the reserves for the dynamic containers are never exceeded. It's acceptable if they do exceed as their memory locality is so large their distance in the pages to load into CPU cache won't matter - it just needs to be a low incidence.
### Shaping Pass
If using the library's cache, `shaper_shape_text_cached` handles the hashing and lookup. As long as a shape is found, it will not enter the uncached code path. By default, this library uses `shaper_shape_harfbuzz` as the `shape_text_uncached` procedure.
Shapes are cached using the following parameters to hash a key:
* font: Font_ID
* font_size: f32
* the text itself: string
All shapers fulfill the following interface:
```odin
Shaper_Shape_Text_Uncached_Proc :: #type proc( ctx : ^Shaper_Context,
atlas : Atlas,
glyph_buffer_size : Vec2,
font : Font_ID,
entry : Entry,
font_px_Size : f32,
font_scale : f32,
text_utf8 : string,
output : ^Shaped_Text
)
```
Which will resolve the output `Shaped_Text`. It has the following definition:
```odin
Shaped_Text :: struct #packed {
glyph : [dynamic]Glyph,
position : [dynamic]Vec2,
visible : [dynamic]i16,
atlas_lru_code : [dynamic]Atlas_Key,
region_kind : [dynamic]Atlas_Region_Kind,
bounds : [dynamic]Range2,
end_cursor_pos : Vec2,
size : Vec2,
font : Font_ID,
px_size : f32,
}
```
The result of the shaping process is the glyphs and their positions for the the shape; historically resembling whats known as a *Slug* of prepared text for printing. The end position of where the user's "cursor" would be is also recorded which provided the end position of the shape. The size of the shape is also resolved here, which if using px_scalar must be downscaled. `measure_shape_size` does the downscaling for the user.
`visible` tracks which of the glyphs will actually be relevant for the draw_list pass. This is to avoid a conditional jump during the draw list gen pass. When accessing glyph or position during the draw_list gen, they will use visible's relative index.
The font and px_size is tracked here as well so they user does not need to provide it to the library's interface and related.
As stated under the main heading of this guide, the the following are within shaped text so that they may be resolved outside of the draw list generation (see: `generate_shape_draw_list`):
* atlas_lru_code
* region_kind
* bounds
These are the same length as the `visible` array, so indexing those will not need to use visibile's relative index.
`shaper_shape_text_latin` does naive shaping by utilizing the codepoint's kern_advance and detecting newlines.
`shaper_shape_harfbuzz` is an actual shaping *engine*. Here is the general idea of how the library utilizes it for shaping:
1. Reset the state of the hb_buffer
2. Determine the line height
3. Go through the codepoints: (for each)
1. Determine the codepoint's script
2. If the script is netural (Uknown, Inherited, or of Common type), the script has not changed, or this is the first codepoint of the shape we can add the codepoint to the buffer.
3. Otherwise we will have to start a shaping run if we do encounter a significant script change. After, we can add the codepoint to the post-run-cleared hb_buffer.
4. This continues until all codepoints have been processed.
4. We do a final shape run after iterating to make sure all codepoints have been processed.
5. Set the size of the shape: X is max line width, Y is line height multiplied by the line count.
6. Resolve the atlas_lru_code, region_kind, and bounds for all visible glyphs
7. Store the font and px_size information.
The `shape_run` procedure within does the following:
1. Setup the buffer for the batch
2. Have harfbuzz shape the buffer
3. Extract glyph infos and positions from the buffer.
4. Iterate through all glyphs
1. If the hb_glyph cluster is > 0, we need to treat it as the indication of a newline glyph. ***(We update position and skip)***
2. Update positioning and other metrics and append output shape's glyph and position.
3. If the glyph is visible we append it to shape's visible (harfbuzz must specify it as not .nodef, and parser must identify it as non-empty)
5. We update the output.end_cursor_pos with the last position processed by the iteration
6. Clear the hb_buffer's contents to prepare for a possible upcoming shape run.
**Note on shape_run.4: The iteration doesn't preserve tracking the clusters, so that information is lost.**
*In the future cluster tracking may be added if its found to be important for high level text features beyond rendering.*
**Note on shape_run.4.1: Don't know if the glyph signifiying newline should be preserved**
See [Harfbuzz documentation](https://harfbuzz.github.io) for additional information.
There are other shapers out there:
* [hamza](https://github.com/saidwho12/hamza): A notable C library that could be setup with bindings.
***Note: Monospace fonts may have a much more trivial shaper (however for fonts with ligatures this may not be the case)***
***They should only need the kern advance of a single glyph as they're all the same. ligatures (I believe) should preserve this kern advance.***
### Draw List Generation
All interface draw text procedures will ultimately call `generate_shape_draw_list`. If the draw procedure is given text, it will call `shaper_shape_text_cached` the text immediately before calling it.
Its implementation uses a batched-pipeline approach where its goal is to populate three arrays behavings as queues:
* oversized: For drawing oversized glyphs
* to_cache: For glyphs that need triangulation & rendering to glyph buffer then blitting to atlas.
* cache: For glyphs that are already cached in the atlas and just need to be blit to the render target.
And then sent those off to `batch_generate_glyphs_draw_list` for further actual generation to be done. The size of a batch is determined by the capacity of the glyph_buffer's `batch_cache`. This can be set in `glyph_draw_params` for startup.
`glyph_buffer.glyph_pack` is utilized by both `generate_shape_draw_list` and `batch_generate_glyphs_draw_list` to various computed data in an SOA data structure for the glyphs.
generate_shape_draw_list outline:
1. Prepare glyph_pack, oversized, to_cache, cached, and reset the batch cache
* `glyph_pack` is resized to to the length of `shape.visible`
* The other arrays populated have their reserved set to that length as well (they will not bounds check capacity on append)
2. Iterate through the shape.visible and resolve glyph_pack's positions.
3. Iterate through shape.visible this time for final region resolution and segregation of glyphs to their appropriate queue.
1. If the glyphs assigned region is `.E` its oversized. The `oversample` used for rendering to render target will either be 2x or 1x depending on how huge it is.
2. The following glyphs are checked to see if their assigned region has the glyph `cached`.
1. If it does, its just appended to cached and marked as seen in the `batch_cache`.
2. If its doesn't then a slot is reserved for within the atlas's region and the glyph is appended to `to_cache`.
3. For either case the atlas_region_bbox is computed.
3. After a batch has been resolved, `batch_generate_glyphs_draw_list` is called.
4. If there is an partially filled batch (the usual case), batch_generate_glyphs_draw_list will be called for it.
5. The cursor_pos is updated with the shape's end cursor position adjusted for the target space.
batch_generate_glyphs_draw_list outline:
The batch is organized into three major stages:
1. glyph transform & draw quads compute
2. glyph_buffer draw list generation (`oversized` & `to_cache`)
3. blit-from-atlas to render target draw list generation (`to_cache` & `cached`)
Glyph transform & draw quads compute does an iteration for each of the 3 arrays.
Nearly all the math for all three is done there *except* for `to_cache`, which does its blitting compute in its glyph_buffer draw-list gen pass.
glyph_buffer draw list generation paths for `oversized` and `to_cache` are unique to each.
For `oversized`:
1. Allocate glyph shapes
2. Iterate oversized:
1. Flush the glyph buffer if flagged todo so (reached glyph allocation limit)
2. Call `generate_glyph_pass_draw_list` for trianglation and rendering to buffer.
3. blit quad.
3. flush the glyph buffer's draw list.
4. free glyph shapes
For `to_cached`:
1. Allocate glyph shapes
2. Iterate to_cache:
1. Flush the glyph buffer if flagged todo so (reached glyph allocation limit)
2. Compute & blit quads for clearing the atlas region and blitting from the buffer to the atlas.
3. Call `generate_glyph_pass_draw_list` for trianglation and rendering to buffer.
3. flush the glyph buffer's draw list.
4. free glyph shapes
5. Do blits from atlas to draw list.
`cached` only needs to blit from the atlas to the render target.
`generate_glyph_pass_draw_list`: sets up the draw call for glyph to the glyph buffer. Currently it also handles triangulation as well. For now the shape triangulation is rudimentary and uses triangle fanning. Eventually it would be nice to offer alternative modes that can be specified on a per-font basis.
`flush_glyph_buffer_draw_list`: Will merge the draw_lists contents of the glyph buffer over to the library's general draw_list, the clear the buffer's draw lists.
### On Layering
The base draw list generation pippline provided by the library allows the user to batch whatever they want into a single "layer".
However, the user most likely would want take into consideration: font instances, font size, colors; these are things that may benefit from having shared locality during a layer batch. Overlaping text benefits from the user to handle the ordering via layers.
Layers (so far) are just a set of offssets tracked by the library's `Context.draw_layer` struct. When `flush_draw_list_layer` is called, the offsets are set to the current length of the draw list. This allows the rendering backend to retrieve the latest set of vertices, indices, and calls to render on a per-layer basis with: `get_draw_list_layer`.
Importantly, this leads to the following pattern when enuquing a layer to render:
1. Begin render pass
2. For codepath that will deal with text layers
1. Process user-level code-path that calls the draw text interface, populating the draw list layer (usually a for loop)
2. After iteration on the layer is complete, render the text layer
1. grab the draw list layer
2. flush the layer so the draw list offsets are reset
3. Repeat until all layers for the codepath are exhausted.
There is consideration to instead explicitly have a draw list with more contextual information of the start and end of each layer. So that batching can be orchestrated in an isolated section of their pipeline.
This would involve just tracking *slices* of thier draw-list that represents layers:
```odin
Draw_List_Layer :: struct {
vertices : []Vertex,
indices : []u32,
calls : []Draw_Call,
}
```
Eventually the library may provide this since adding that feature is relatively cheap and and a low line-count addition to the interface.
There should be little to no perfomrance loss from doing so as the iteration size is two large of a surface area to matter (so its just pipeline ergonomics)

68
code/vefontcache/docs/guide_backend.md Normal file
View File

@@ -0,0 +1,68 @@
# Guide: Backend
The end-user needs to adapt this library to hook into their own codebase. For reference, they can check the [examples](../examples/) and [backend](../backend/) directories for working code that demonstrates what this guide covers.
When rendering text, users need to handle two main aspects: the text to draw and its "layering". Similar to UIs, text should be drawn in layer batches, where each layer can represent a pass with arbitrary distinctions from other layers.
The following components are required:
* Vertex and Index Buffers for glyph meshes
* Glyph shader for rendering glyphs to the glyph buffer
* Atlas shader for blitting upscaled glyph quads from the glyph buffer to an atlas region slot (downsampled)
* "Screen or Target" shader for blitting glyph quads from the atlas to a render target or swapchain
* The glyph, atlas, and target image buffers
Currently, the library doesn't support sub-pixel AA, so we're only rendering to R8 images.
## Rendering Passes
There are four passes that need to be handled when rendering a draw list:
* Glyph: Rendering a glyph mesh to the glyph buffer
* Atlas: Blitting a glyph quad from the glyph buffer to an atlas slot
* Target: Blitting from the atlas image to the target image
* Target_Uncached: Blitting from the glyph buffer image to the target image
The Target & Target_Uncached passes can technically be handled in the same case. The user just needs to swap between using the atlas image and the glyph buffer image. This is how the backend_soko.odin's `render_text_layer` has these passes set up.
## Vertex Buffer Layout
The vertex buffer has the following layout for all passes:
* `[2]f32` for positions
* `[2]f32` for texture coords (Offset is naturally `[2]f32`)
* Total stride: `[4]f32`
---
The index buffer is a simple u32 stream.
For quad mesh layout details, see `blit_quad` in [draw.odin](../vefontcache/draw.odin).
For glyph shape triangulation meshes, the library currently only uses a triangle fanning technique, implemented in `fill_path_via_fan_triangulation` within [draw.odin](../vefontcache/draw.odin). Eventually, the library will support other modes on a per-font basis.
## UV Coordinate Conventions (GLSL vs HLSL)
DirectX, Metal, and Vulkan consider the top-left corner as (0, 0), where the Y axis increases downward (traditional screenspace). This library follows OpenGL's convention, where (0, 0) is at the bottom-left (Y goes up).
Adjust the UV coordinates in your shader accordingly:
```c
#if !OpenGL
uv = vec2(v_texture.x, 1.0 - v_texture.y);
#else
uv = vec2(v_texture.x, v_texture.y);
#endif
```
Eventually, the library will support both conventions as a comp-time conditional.
## Retrieving & Processing the layer
`get_draw_list_layer` will provide the layer's vertex, index, and draw call slices. Unless the default is overwritten, it will call `optimize_draw_list` before returning the slices (profile to see whats better for your use case).
Once those are retrived, call `flush_draw_list_layer` to update the layer offsets tracked by the library's `Context`.
The vertex and index slices just needed to be appended to your backend's vertex and index buffers.
The draw calls need to be iterated with a switch statement for the aforementioned pass types. Within the case you can construct the enqueue the passes.
---

17
code/vefontcache/docs/original/LICENSE.md Normal file
View File

@@ -0,0 +1,17 @@
Vertex Engine GPU Font Cache
Copyright 2020 Xi Chen
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
associated documentation files (the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge, publish, distribute,
sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial
portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT
NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.