Finished inital draft for draw list generation docs in guide_architecture.md

Readme.md

@@ -33,13 +33,17 @@ Upcoming:
 
 * Support for ear-clipping triangulation, or just better triangulation..
   * Support for which triangulation method used on a by font basis?
-  * https://www.microsoft.com/en-us/research/wp-content/uploads/2005/01/p1000-loop.pdf
+  * [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2005/01/p1000-loop.pdf)
 * Multi-threading supported job queue.
   * Lift heavy-lifting portion of the library's context into a thread context.
   * Synchronize threads by merging their generated layered draw list into a finished draw-list for processing on the user's render thread.
   * User defines how thread context's are distributed for drawing (a basic quandrant based selector procedure will be provided.)
 
-See: [docs/Readme.md](docs/Readme.md) for the library's interface.
+## Documentation
+
+* [docs/Readme.md](docs/Readme.md) for the library's interface.
+* [docs/guide_backend.md](docs/guide_backend.md) for information on whats needed rolling your own backend.
+* [docs/guide_architecture.md](docs/guide_architecture.md) for an in-depth breakdown of the significant design decisions, and codepaths.
 
 ## Building
 

backend/sokol/blit_atlas.shdc.glsl

@@ -35,7 +35,6 @@ float down_sample_to_texture( vec2 uv, vec2 texture_size )
 
 void main()
 {
-	// const vec2 texture_size = 1.0f /  vec2( 2048.0f, 512.0f );
 	const vec2 texture_size = 1.0f / glyph_buffer_size;
 	if ( region == 0 || region == 1 || region == 2 || region == 4 )
 	{

backend/sokol/draw_text.shdc.glsl

@@ -32,7 +32,6 @@ void main()
 {
 	float alpha = texture(sampler2D( ve_draw_text_src_texture, ve_draw_text_src_sampler ), uv ).x;
 
-	// const vec2  texture_size = 1.0f / vec2( 2048.0f, 512.0f );
 	const vec2  texture_size = glyph_buffer_size;
 	const float down_sample  = 1.0f / over_sample;
 

docs/guide_architecture.md

@@ -4,7 +4,7 @@ Overview on the state of package design and codepath layout.
 
 ---
 
-The purpose of this library to really allieviate four issues with one encapsulation:
+The purpose of this library to really allieviate four issues with one encapsulating package:
 
 * font parsing
 * text codepoint shaping
@@ -45,7 +45,7 @@ Shapes are cached using the following parameters to hash a key:
 * font_size: f32
 * the text itself: string
 
-All shapers fullfill the following interface:
+All shapers fulfill the following interface:
 
 ```odin
 Shaper_Shape_Text_Uncached_Proc :: #type proc( ctx : ^Shaper_Context,
@@ -60,7 +60,7 @@ Shaper_Shape_Text_Uncached_Proc :: #type proc( ctx : ^Shaper_Context,
 )
 ```
 
-Which will resolve the output `Shaped_Text`. Which has the following structure:
+Which will resolve the output `Shaped_Text`. It has the following definition:
 
 ```odin
 Shaped_Text :: struct #packed {
@@ -91,8 +91,7 @@ As stated under the main heading of this guide, the the following are within sha
 
 They're arrays are the same length as `visible`, so indexing those will not need to use visibile's relative index.
 
-`shaper_shape_text_latin` does naive shaping by utilizing the codepoint's kern_advance and detecting newlines.
+`shaper_shape_text_latin` does naive shaping by utilizing the codepoint's kern_advance and detecting newlines.  
-
 `shaper_shape_harfbuzz` is an actual shaping *engine*. Here is the general idea of how the library utilizes it for shaping:
 
 1. Reset the state of the hb_buffer
@@ -100,7 +99,7 @@ They're arrays are the same length as `visible`, so indexing those will not need
 3. Go through the codepoints: (for each)
     1. Determine the codepoint's script
     2. If the script is netural (Uknown, Inherited, or of Common type), or the script has not changed, or this is the first codepoint of the shape we can add the codepoint to the buffer.
-    3. Otherwise we may have to start a shaping run if we do encounter a significant script change. After we can add the codepoint to the post-run-cleared hb_buffer.
+    3. Otherwise we may have to start a shaping run if we do encounter a significant script change. After, we can add the codepoint to the post-run-cleared hb_buffer.
     4. This continues until all codepoints have been processed.
 4. We do a final shape run after iterating to make sure all codepoints have been processed.
 5. Set the size of the shape: x is max line width, y is line height multiplied by the line count.
@@ -135,14 +134,80 @@ There are other shapers out there:
 
 ### Draw List Generation
 
+All interface draw text procedures will ultimately call: `generate_shape_draw_list`. If the draw procedure is given text, it will call `shaper_shape_text_cached` the text immeidately before calling it.
 
+Its implementation uses a batched-pipeline approach where its goal is to populate three arrays behavings as queues:  
+
+* oversized: For drawing oversized glyphs
+* to_cache: For glyphs that need triangulation/rendering to glyph buffer then blitting to atlas.
+* cache: For glyphs that are already cached in the atlas and just need to be blit to the render target.
+
+And then sent those off to `batch_generate_glyphs_draw_list` for futher actual generaiton to be done. The size of a batch is determined by the capacity of the glyph_buffer's `batch_cache`. This can be set in `glyph_draw_params` for startup.
+
+`glyph_buffer.glyph_pack` is utilized by both `generate_shape_draw_list` and `batch_generate_glyphs_draw_list` to various computed data in an SOA data structure for the glyphs.
+
+generate_shape_draw_list outline:
+
+1. Prepare glyph_pack, oversized, to_cache, cached, and reset the batch cache
+    * `glyph_pack` is resized to to the length of `shape.visible`
+    * The other arrays populated have their reserved set to that length as well (they will not bounds check capacity on append)
+2. Iterate though the shape.visible and resolve glyph_pack's positions.
+3. Iterate through shape.visible this time for final region resolution and segregation of glyphs to their appropriate queue.
+    1. If the glyphs assigned region is `.E` its oversized. The `oversample` used for rendering to render target will either be 2x or 1x depending on how huge it is.
+    2. The following glyphs are checked to see if their assigned region has the glyph `cached`.
+        1. If it does, its just appended to cached and marked as seen in the `batch_cache`.
+        2. If its doesn't then a slot is reseved for within the atlas's region and the glyph is appended to `to_cache`.
+        3. For either case the atlas_region_bbox is computed.
+    3. After a batch has been resolved, `batch_generate_glyphs_draw_list` is called.
+4. If there is an partially filled batch (the usual case), batch_generate_glyphs_draw_list will be called for it.
+5. The cursor_pos is updated with the shape's end cursor position adjusted for the target space.
+
+batch_generate_glyphs_draw_list outline:
+
+The batch is organized into three major stages:
+
+1. glyph transform & draw quads compute
+2. glyph_buffer draw list generation (`oversized` & `to_cache`)
+3. blit-from-atlas to render target draw list generation (`to_cache` & `cached`)
+
+Glyph transform & draw quads compute does an iteration for each of the 3 arrays.  
+Nearly all the math for all three is there *except* for `to_cache`, which does its blitting compute in its glyph_buffer draw-list gen pass.
+
+glyph_buffer draw list generation paths for `oversized` and `to_cache` are unique to each.
+
+For `oversized`:
+
+1. Allocate glyph shapes
+2. Iterate oversized:
+    1. Flush the glyph buffer if flagged todo so (reached glyph allocation limit)
+    2. Call `generate_glyph_pass_draw_list` for trianglation and rendering to buffer.
+    3. blit quad.
+3. flush the glyph buffer's draw list.
+4. free glyph shapes
+
+For `to_cached`:
+
+1. Allocate glyph shapes
+2. Iterate to_cache:
+    1. Flush the glyph buffer if flagged todo so (reached glyph allocation limit)
+    2. Compute & blit quads for clearing the atlas region and blitting from the buffer to the atlas.
+    3. Call `generate_glyph_pass_draw_list` for trianglation and rendering to buffer.
+3. flush the glyph buffer's draw list.
+4. free glyph shapes
+5. Do blits from atlas to draw list.
+
+`cached` only needsto blit from the atlas to the render target.
+
+`generate_glyph_pass_draw_list`: sets up the draw call for glyph to the glyph buffer. Currently it also handles triangulation as well. For now the shape triangulation is rudimentary and uses triangle fanning. Eventually it would be nice to offer alternatve modes that can be specified on a per-font basis.
+
+`flush_glyph_buffer_draw_list`: Will merge the draw_lists contents of the glyph buffer over to the library's general draw_list, the clear the buffer's draw lists.
 
 ### On Layering
 
 The base draw list generation pippline provided by the library allows the user to batch whatever the want into a single "layer".
 However, the user most likely would want take into consideration: font instances, font size, colors; these are things that may benefit from having shared locality during a layer batch. Overlaping text benefits from the user to handle the ordering via layers.
 
-Layers (so far) are just a set of offssets tracked by the library's `Context.draw_layer` struct. When `flush_draw_list_layer` is called, the offsets are set to the current leng of the draw list. This allows the rendering backend to retrieve the latest set of vertices, indices, and calls to render on a per-layer basis with: `get_draw_list_layer`. 
+Layers (so far) are just a set of offssets tracked by the library's `Context.draw_layer` struct. When `flush_draw_list_layer` is called, the offsets are set to the current leng of the draw list. This allows the rendering backend to retrieve the latest set of vertices, indices, and calls to render on a per-layer basis with: `get_draw_list_layer`.
 
 Importantly, this leads to the following pattern when enuquing a layer to render:
 
@@ -154,7 +219,7 @@ Importantly, this leads to the following pattern when enuquing a layer to render
         2. flush the layer so the draw list offsets are reset
     3. Repeat until all layers for the codepath are exhausted.
 
-There is consideration to instead explicitly have a draw list with more contextual information of the start and end of each layer. So that batching can be orchestrated in a section of their pipline.
+There is consideration to instead explicitly have a draw list with more contextual information of the start and end of each layer. So that batching can be orchestrated in an isolated section of their pipline.
 
 This would involve just tracking *slices* of thier draw-list that represents layers:
 

vefontcache/draw.odin

@@ -28,23 +28,14 @@ Glyph_Draw_Quad :: struct {
 // This is used by generate_shape_draw_list & batch_generate_glyphs_draw_list 
 // to track relevant glyph data in soa format for pipelined processing
 Glyph_Pack_Entry :: struct #packed {
-	vis_index          : i16,
 	position           : Vec2,
-
 	atlas_index        : i32,
-	in_atlas           : b8,
-	should_cache       : b8,
 	region_pos         : Vec2,
 	region_size        : Vec2,
-
 	over_sample        : Vec2, // Only used for oversized glyphs
-
 	shape              : Parser_Glyph_Shape,
 	draw_transform     : Transform,
-
 	draw_quad          : Glyph_Draw_Quad,
-	draw_atlas_quad    : Glyph_Draw_Quad,
-	draw_quad_clear    : Glyph_Draw_Quad,
 	buffer_x           : f32,
 	flush_glyph_buffer : b8,
 }
@@ -322,6 +313,16 @@ generate_shape_draw_list :: proc( draw_list : ^Draw_List, shape : Shaped_Text,
 	to_cache   := & glyph_buffer.to_cache
 	cached     := & glyph_buffer.cached
 	resize_soa_non_zero(glyph_pack, len(shape.visible))
+	
+	profile_begin("batching & segregating glyphs")
+	// We do any reservation up front as appending to the array's will not check.
+	reserve(oversized, len(shape.visible))
+	reserve(to_cache,  len(shape.visible))
+	reserve(cached,    len(shape.visible))
+	clear(oversized)
+	clear(to_cache)
+	clear(cached)
+	reset_batch( & glyph_buffer.batch_cache)
 
 	append_sub_pack :: #force_inline proc ( pack : ^[dynamic]i32, entry : i32 )
 	{
@@ -340,16 +341,6 @@ generate_shape_draw_list :: proc( draw_list : ^Draw_List, shape : Shaped_Text,
 	}
 	profile_end()
 
-	profile_begin("batching & segregating glyphs")
-	// We do any reservation up front as appending to the array's will not check.
-	reserve(oversized, len(shape.visible))
-	reserve(to_cache,  len(shape.visible))
-	reserve(cached,    len(shape.visible))
-	clear(oversized)
-	clear(to_cache)
-	clear(cached)
-	reset_batch( & glyph_buffer.batch_cache)
-
 	for & glyph, index in glyph_pack
 	{
 		// atlas_lru_code, region_kind, and bounds are all 1:1 with shape.visible
@@ -616,12 +607,6 @@ batch_generate_glyphs_draw_list :: proc ( draw_list : ^Draw_List,
 				& glyph_buffer.allocated_x
 			)
 
-			// vis_id := shape.visible[id]
-			// error : Allocator_Error
-			// glyph.shape, error = parser_get_glyph_shape(entry.parser_info, shape.glyph[vis_id])
-			// assert(error == .None)
-			// assert(len(glyph.shape) > 0)
-			
 			generate_glyph_pass_draw_list( draw_list, & glyph_buffer.shape_gen_scratch,
 				glyph_pack[id].shape, 
 				entry.curve_quality, 
@@ -630,9 +615,6 @@ batch_generate_glyphs_draw_list :: proc ( draw_list : ^Draw_List,
 				glyph_pack[id].draw_transform.scale
 			)
 
-			// assert(len(glyph.shape) > 0)
-			// parser_free_shape(entry.parser_info, glyph.shape)
-
 			target_quad := & glyph_pack[id].draw_quad
 
 			draw_to_target : Draw_Call
@@ -743,12 +725,6 @@ batch_generate_glyphs_draw_list :: proc ( draw_list : ^Draw_List,
 			append( & glyph_buffer.clear_draw_list.calls, clear_target_region )
 			append( & glyph_buffer.draw_list.calls,       blit_to_atlas )
 
-			// vis_id := shape.visible[id]
-			// error : Allocator_Error
-			// glyph.shape, error = parser_get_glyph_shape(entry.parser_info, shape.glyph[vis_id])
-			// assert(error == .None)
-			// assert(len(glyph.shape) > 0)
-	
 			// Render glyph to glyph render target (FBO)
 			generate_glyph_pass_draw_list( draw_list, & glyph_buffer.shape_gen_scratch, 
 				glyph.shape, 
@@ -757,9 +733,6 @@ batch_generate_glyphs_draw_list :: proc ( draw_list : ^Draw_List,
 				glyph.draw_transform.pos, 
 				glyph.draw_transform.scale 
 			)
-			
-			// assert(len(glyph.shape) > 0)
-			// parser_free_shape(entry.parser_info, glyph.shape)
 		}
 
 		flush_glyph_buffer_draw_list(draw_list, & glyph_buffer.draw_list, & glyph_buffer.clear_draw_list, & glyph_buffer.allocated_x)

vefontcache/vefontcache.odin

@@ -697,7 +697,12 @@ shape_text_uncached :: #force_inline proc( ctx : ^Context, font : Font_ID, px_si
     <-> scale   : Scale the glyph beyond its default scaling from its px_size.
 */
 @(optimization_mode="favor_size")
-draw_text_shape_normalized_space :: #force_inline proc( ctx : ^Context, colour : RGBAN,  position : Vec2, scale : Vec2,  shape  : Shaped_Text )
+draw_text_shape_normalized_space :: #force_inline proc( ctx : ^Context, 
+	colour   : RGBAN,  
+	position : Vec2, 
+	scale    : Vec2,  
+	shape    : Shaped_Text
+)
 {
 	profile(#procedure)
 	assert( ctx != nil )