extend eval viz pipeline with split between edit/display expression strings per-block and per-row. this allows reconstruction of the string that would lead to the same eval as a row (even if it is auto-generated by e.g. expanding a root-level row; also allow -- to extend eval expressions with passthrough text, used for view rules

This commit is contained in:
Ryan Fleury
2024-04-01 13:55:10 -07:00
parent cfb34ec560
commit 835eae208a
7 changed files with 71 additions and 26 deletions
+8
View File
@@ -414,6 +414,14 @@ eval_token_array_from_text(Arena *arena, String8 text)
eval_token_chunk_list_push(scratch.arena, &tokens, 256, &token);
}
// rjf: symbolic strings matching `--` mean the remainder of the string
// is reserved for external usage. the rest of the stream should not
// be tokenized.
else if(idx == active_token_start_idx+2 && text.str[active_token_start_idx] == '-' && text.str[active_token_start_idx+1] == '-')
{
break;
}
// rjf: if we got a symbol string of N>1 characters, then we need to
// apply the maximum-munch rule, and produce M<=N tokens, where each
// formed token is the maximum size possible, given the legal