Separate the I18N calls for immutable strings and for pluraliseable strings.

Also update tests.
This commit is contained in:
Maurizio M. Gavioli
2024-06-19 10:10:26 +02:00
parent 7064166da2
commit 53755824fb
3 changed files with 177 additions and 77 deletions
+33 -19
View File
@@ -1,31 +1,44 @@
/* /*
The `i18n` package is flexible and easy to use. The `i18n` package is a flexible and easy to use way to localise applications.
It has one call to get a translation: `get`, which the user can alias into something like `T`. It has two calls to get a translation: `get()` amd `get_n()`, which the user can alias into something like `T` and `Tn`
with statements like:
T :: i18n.get
Tn :: i18n.get_n.
`get`, referred to as `T` here, has a few different signatures. `get()` is used for retrieving the translation of sentences which **never**change in form,
All of them will return the key if the entry can't be found in the active translation catalog. like for instance "Connection established" or "All temporary files have been deleted".
Note that the number (singular, dual, plural, whatever else) is not relevant: the semtece is fixed and it will have only one possible translation in any other language.
- `T(key)` returns the translation of `key`. `get_n()` is used for retrieving the translations of sentences which change according to the number of items referenced.
- `T(key, n)` returns a pluralized translation of `key` according to value `n`. The various signatures of `get_n()` have one more parameter, `n`, which will receive that number,
and which be used to select the correct form according to the pluraliser attached to the message catalogue when initially loaded;
for instance, to summarise a rather complex matter, some languages use the singular form when reerring to 0 items and some use the (only in their case) plural forms;
also, languages may have more or less quantifier forms than a single singular form an a universal plural form:
for istance, Chinese has just one form for any quantity, while Welsh may have up to 6 different forms for specific different quantities.
- `T(section, key)` returns the translation of `key` in `section`. Both `get()` and `get_n()`, referred to as `T` and `Tn` here, have several different signatures.
- `T(section, key, n)` returns a pluralized translation of `key` in `section` according to value `n`. All of them will return the key if the entry can't be found in the active translation catalogue.
By default lookup take place in the global `i18n.ACTIVE` catalogue for ease of use, unless a speciic catalogue is supplied
By default lookup take place in the global `i18n.ACTIVE` catalog for ease of use. - `T(key)` returns the translation of `key`.
If you want to override which translation to use, for example in a language preview dialog, you can use the following: - `T(key, catalog)` returns the translation of `key` from explictly supplied catalogue.
- `T(section, key)` returns the translation of `key` in `section`.
- `T(section, key, catalog)` returns the translation of `key` in `section` from explictly supplied catalogue.
- `T(key, n, catalog)` returns the pluralized version of `key` from explictly supplied catalog. - `Tn(key, n)` returns the translation of `key` according to number of items `n`.
- `T(section, key, n, catalog)` returns the pluralized version of `key` in `section` from explictly supplied catalog. - `Tn(key, n, catalog)` returns the translation of `key` from explictly supplied catalogue.
- `Tn(section, key, n)` returns the translation of `key` in `section` according to number of items `n`.
- `Tn(section, key, n, catalog)` returns the translation of `key` in `section` according to number of items `n` from explictly supplied catalogue.
If a catalog has translation contexts or sections, then omitting it in the above calls looks up in section "". If a catalog has translation contexts or sections, then omitting it in the above calls looks up in section "".
The default pluralization rule is n != 1, which is to say that passing n == 1 (or not passing n) returns the singular form. The default pluralization rule is `n != 1``, which is to say that passing n == 1 returns the singular form (in slot 0).
Passing n != 1 returns plural form 1. Passing n != 1 returns plural form in slot 1 (if any).
Should a language not conform to this rule, you can pass a pluralizer procedure to the catalog parser. Should a language not conform to this rule, you can pass a pluralizer procedure to the catalog parser.
This is a procedure that maps an integer to an integer, taking a value and returning which plural slot should be used. This is a procedure that maps an integer to an integer, taking a numbe of item and returning which plural slot should be used.
You can also assign it to a loaded catalog after parsing, of course. You can also assign it to a loaded catalog after parsing, of course.
@@ -35,6 +48,7 @@ Example:
import "core:text/i18n" import "core:text/i18n"
T :: i18n.get T :: i18n.get
Tn :: i18n.get_n
mo :: proc() { mo :: proc() {
using fmt using fmt
@@ -60,9 +74,9 @@ Example:
println(T("Hellope, World!")) println(T("Hellope, World!"))
println("-----") println("-----")
// We pass 1 into `T` to get the singular format string, then 1 again into printf. // We pass 1 into `T` to get the singular format string, then 1 again into printf.
printf(T("There is %d leaf.\n", 1), 1) printf(Tn("There is %d leaf.\n", 1), 1)
// We pass 42 into `T` to get the plural format string, then 42 again into printf. // We pass 42 into `T` to get the plural format string, then 42 again into printf.
printf(T("There is %d leaf.\n", 42), 42) printf(Tn("There is %d leaf.\n", 42), 42)
/* /*
This isn't in the translation catalog, so the key is passed back untranslated. This isn't in the translation catalog, so the key is passed back untranslated.
@@ -99,8 +113,8 @@ Example:
println("-----") println("-----")
println("--- apple_count section ---") println("--- apple_count section ---")
println("apple_count:%d apple(s) =") println("apple_count:%d apple(s) =")
println("\t 1 =", T("apple_count", "%d apple(s)", 1)) println("\t 1 =", Tn("apple_count", "%d apple(s)", 1))
println("\t 42 =", T("apple_count", "%d apple(s)", 42)) println("\t 42 =", Tn("apple_count", "%d apple(s)", 42))
} }
*/ */
package i18n package i18n
+122 -38
View File
@@ -84,64 +84,149 @@ DEFAULT_PARSE_OPTIONS :: Parse_Options{
merge_sections = false, merge_sections = false,
} }
/* // *****************
Several ways to use: // get() PROC GROUP
- get(key), which defaults to the singular form and i18n.ACTIVE catalog, or // *****************
- get(key, number), which returns the appropriate plural from the active catalog, or
- get(key, number, catalog) to grab text from a specific one.
*/
get_single_section :: proc(key: string, number := 1, catalog: ^Translation = ACTIVE) -> (value: string) {
/*
A lot of languages use singular for 1 item and plural for 0 or more than 1 items. This is our default pluralize rule.
*/
plural := 1 if number != 1 else 0
if catalog.pluralize != nil { /*
plural = catalog.pluralize(number) Returns the first translation string for the passed `key`.
} It is also aliases with `get( )`.
return get_by_slot(key, plural, catalog)
Two ways to use it:
- get(key), which defaults to the `i18n.ACTIVE`` catalogue, or
- get(key, catalog) to grab text from a specific loaded catalogue.
Inputs:
- key: the string to translate.
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/
get_single_section :: proc(key: string, catalog: ^Translation = ACTIVE) -> (value: string) {
return get_by_slot(key, 0, catalog)
} }
/* /*
Several ways to use: Returns the first translation string for the passed `key` in a specific section or context.
- get(section, key), which defaults to the singular form and i18n.ACTIVE catalog, or It is also aliases with `get( )`.
- get(section, key, number), which returns the appropriate plural from the active catalog, or
- get(section, key, number, catalog) to grab text from a specific one.
*/
get_by_section :: proc(section, key: string, number := 1, catalog: ^Translation = ACTIVE) -> (value: string) {
/*
A lot of languages use singular for 1 item and plural for 0 or more than 1 items. This is our default pluralize rule.
*/
plural := 1 if number != 1 else 0
if catalog.pluralize != nil { Two ways to use it:
plural = catalog.pluralize(number) - get(section, key), which defaults to the `i18n.ACTIVE`` catalogue, or
} - get(section, key, catalog) to grab text from a specific loaded catalogue.
return get_by_slot(section, key, plural, catalog)
Inputs:
- section: the catalogue section (sometime also called 'context') from which to lookup the translation
- key: the string to translate.
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/
get_by_section :: proc(section, key: string, catalog: ^Translation = ACTIVE) -> (value: string) {
return get_by_slot(section, key, 0, catalog)
} }
get :: proc{get_single_section, get_by_section} get :: proc{get_single_section, get_by_section}
// *****************
// get_n() PROC GROUP
// *****************
/* /*
Several ways to use: Returns the translation string for the passed `key` in a specific plural form (if present in the catalogue).
- get_by_slot(key), which defaults to the singular form and i18n.ACTIVE catalog, or It is also aliases with `get_n( )`.
- get_by_slot(key, slot), which returns the requested plural from the active catalog, or
- get_by_slot(key, slot, catalog) to grab text from a specific one. Two ways to use it:
- get_n(key, quantity), which returns the appropriate plural from the active catalogue, or
- get_n(key, quantity, catalog) to grab text from a specific loaded catalogue.
Inputs:
- key: the string to translate.
- qantity: the quantity of item to be used to select the correct plural form.
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/
get_single_section_w_plural :: proc(key: string, quantity: int, catalog: ^Translation = ACTIVE) -> (value: string) {
/*
A lot of languages use singular for 1 item and plural for 0 or more than 1 items. This is our default pluralize rule.
*/
slot := 1 if quantity != 1 else 0
if catalog.pluralize != nil {
slot = catalog.pluralize(quantity)
}
return get_by_slot(key, slot, catalog)
}
/*
Returns the translation string for the passed `key` in a specific plural form (if present in the catalogue)
in a specific section or context.
It is also aliases with `get_n( )`.
Two ways to use it:
- get(section, key, quantity), which returns the appropriate plural from the active catalogue, or
- get(section, key, quantity, catalog) to grab text from a specific loaded catalogue.
Inputs:
- section: the catalogue section (sometime also called 'context') from which to lookup the translation
- key: the string to translate.
- qantity: the quantity of item to be used to select the correct plural form.
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/
get_by_section_w_plural :: proc(section, key: string, quantity: int, catalog: ^Translation = ACTIVE) -> (value: string) {
/*
A lot of languages use singular for 1 item and plural for 0 or more than 1 items. This is our default pluralize rule.
*/
slot := 1 if quantity != 1 else 0
if catalog.pluralize != nil {
slot = catalog.pluralize(quantity)
}
return get_by_slot(section, key, slot, catalog)
}
get_n :: proc{get_single_section_w_plural, get_by_section_w_plural}
// *****************
// get_by_slot() PROC GROUP
// *****************
/*
Two ways to use:
- get_by_slot(key, slot), which returns the requested plural from the active catalogue, or
- get_by_slot(key, slot, catalog) to grab text from a specific loaded catalogue.
If a file format parser doesn't (yet) support plural slots, each of the slots will point at the same string. If a file format parser doesn't (yet) support plural slots, each of the slots will point at the same string.
- section: the catalogue section (sometime also called 'context') from which to lookup the translation
Inputs:
- key: the string to translate.
- qantity: the translation slot to choose (slots refer to plural forms specific for each language and their meaning changes from catalogue to catalogue).
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/ */
get_by_slot_single_section :: proc(key: string, slot := 0, catalog: ^Translation = ACTIVE) -> (value: string) { get_by_slot_single_section :: proc(key: string, slot: int, catalog: ^Translation = ACTIVE) -> (value: string) {
return get_by_slot_by_section("", key, slot, catalog) return get_by_slot_by_section("", key, slot, catalog)
} }
/* /*
Several ways to use: Two ways to use:
- get_by_slot(key), which defaults to the singular form and i18n.ACTIVE catalog, or
- get_by_slot(key, slot), which returns the requested plural from the active catalog, or - get_by_slot(key, slot), which returns the requested plural from the active catalog, or
- get_by_slot(key, slot, catalog) to grab text from a specific one. - get_by_slot(key, slot, catalog) to grab text from a specific one.
If a file format parser doesn't (yet) support plural slots, each of the slots will point at the same string. If a file format parser doesn't (yet) support plural slots, each of the slots will point at the same string.
Inputs:
- section: the catalogue section (sometime also called 'context') from which to lookup the translation
- key: the string to translate.
- qantity: the translation slot to choose (slots refer to plural forms specific for each language and their meaning changes from catalogue to catalogue).
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/ */
get_by_slot_by_section :: proc(section, key: string, slot := 0, catalog: ^Translation = ACTIVE) -> (value: string) { get_by_slot_by_section :: proc(section, key: string, slot: int, catalog: ^Translation = ACTIVE) -> (value: string) {
if catalog == nil || section not_in catalog.k_v { if catalog == nil || section not_in catalog.k_v {
/* /*
Return the key if the catalog catalog hasn't been initialized yet, or the section is not present. Return the key if the catalog catalog hasn't been initialized yet, or the section is not present.
@@ -161,7 +246,6 @@ get_by_slot_by_section :: proc(section, key: string, slot := 0, catalog: ^Transl
get_by_slot :: proc{get_by_slot_single_section, get_by_slot_by_section} get_by_slot :: proc{get_by_slot_single_section, get_by_slot_by_section}
/* /*
Same for destroy:
- destroy(), to clean up the currently active catalog catalog i18n.ACTIVE - destroy(), to clean up the currently active catalog catalog i18n.ACTIVE
- destroy(catalog), to clean up a specific catalog. - destroy(catalog), to clean up a specific catalog.
*/ */
+21 -19
View File
@@ -5,6 +5,7 @@ import "core:testing"
import "core:text/i18n" import "core:text/i18n"
T :: i18n.get T :: i18n.get
Tn :: i18n.get_n
Test :: struct { Test :: struct {
section: string, section: string,
@@ -47,7 +48,8 @@ test_custom_pluralizer :: proc(t: ^testing.T) {
{"", "Message1/plural", "This is message 1", 1}, {"", "Message1/plural", "This is message 1", 1},
{"", "Message1/plural", "This is message 1 - plural A", 1_000_000}, {"", "Message1/plural", "This is message 1 - plural A", 1_000_000},
{"", "Message1/plural", "This is message 1 - plural B", 42}, {"", "Message1/plural", "This is message 1 - plural B", 42},
// This isn't in the catalog, so should ruturn the key.
// This isn't in the catalog, so should return the key.
{"", "Come visit us on Discord!", "Come visit us on Discord!", 1}, {"", "Come visit us on Discord!", "Come visit us on Discord!", 1},
}, },
}) })
@@ -61,11 +63,11 @@ test_mixed_context :: proc(t: ^testing.T) {
plural = nil, plural = nil,
tests = { tests = {
// These are in the catalog. // These are in the catalog.
{"", "Message1", "This is message 1 without Context", 1}, {"", "Message1", "This is message 1 without Context",-1},
{"Context", "Message1", "This is message 1 with Context", 1}, {"Context", "Message1", "This is message 1 with Context", -1},
// This isn't in the catalog, so should ruturn the key. // This isn't in the catalog, so should ruturn the key.
{"", "Come visit us on Discord!", "Come visit us on Discord!", 1}, {"", "Come visit us on Discord!", "Come visit us on Discord!", -1},
}, },
}) })
} }
@@ -90,15 +92,15 @@ test_nl_mo :: proc(t: ^testing.T) {
plural = nil, // Default pluralizer plural = nil, // Default pluralizer
tests = { tests = {
// These are in the catalog. // These are in the catalog.
{"", "There are 69,105 leaves here.", "Er zijn hier 69.105 bladeren.", 1}, {"", "There are 69,105 leaves here.", "Er zijn hier 69.105 bladeren.", -1},
{"", "Hellope, World!", "Hallo, Wereld!", 1}, {"", "Hellope, World!", "Hallo, Wereld!", -1},
{"", "There is %d leaf.\n", "Er is %d blad.\n", 1}, {"", "There is %d leaf.\n", "Er is %d blad.\n", 1},
{"", "There are %d leaves.\n", "Er is %d blad.\n", 1}, {"", "There are %d leaves.\n", "Er is %d blad.\n", 1},
{"", "There is %d leaf.\n", "Er zijn %d bladeren.\n", 42}, {"", "There is %d leaf.\n", "Er zijn %d bladeren.\n", 42},
{"", "There are %d leaves.\n", "Er zijn %d bladeren.\n", 42}, {"", "There are %d leaves.\n", "Er zijn %d bladeren.\n", 42},
// This isn't in the catalog, so should ruturn the key. // This isn't in the catalog, so should ruturn the key.
{"", "Come visit us on Discord!", "Come visit us on Discord!", 1}, {"", "Come visit us on Discord!", "Come visit us on Discord!", -1},
}, },
}) })
} }
@@ -111,15 +113,15 @@ test_qt_linguist :: proc(t: ^testing.T) {
plural = nil, // Default pluralizer plural = nil, // Default pluralizer
tests = { tests = {
// These are in the catalog. // These are in the catalog.
{"Page", "Text for translation", "Tekst om te vertalen", 1}, {"Page", "Text for translation", "Tekst om te vertalen", -1},
{"Page", "Also text to translate", "Ook tekst om te vertalen", 1}, {"Page", "Also text to translate", "Ook tekst om te vertalen", -1},
{"installscript", "99 bottles of beer on the wall", "99 flessen bier op de muur", 1}, {"installscript", "99 bottles of beer on the wall", "99 flessen bier op de muur", -1},
{"apple_count", "%d apple(s)", "%d appel", 1}, {"apple_count", "%d apple(s)", "%d appel", 1},
{"apple_count", "%d apple(s)", "%d appels", 42}, {"apple_count", "%d apple(s)", "%d appels", 42},
// These aren't in the catalog, so should ruturn the key. // These aren't in the catalog, so should ruturn the key.
{"", "Come visit us on Discord!", "Come visit us on Discord!", 1}, {"", "Come visit us on Discord!", "Come visit us on Discord!", -1},
{"Fake_Section", "Come visit us on Discord!", "Come visit us on Discord!", 1}, {"Fake_Section", "Come visit us on Discord!", "Come visit us on Discord!", -1},
}, },
}) })
} }
@@ -133,16 +135,16 @@ test_qt_linguist_merge_sections :: proc(t: ^testing.T) {
options = {merge_sections = true}, options = {merge_sections = true},
tests = { tests = {
// All of them are now in section "", lookup with original section should return the key. // All of them are now in section "", lookup with original section should return the key.
{"", "Text for translation", "Tekst om te vertalen", 1}, {"", "Text for translation", "Tekst om te vertalen", -1},
{"", "Also text to translate", "Ook tekst om te vertalen", 1}, {"", "Also text to translate", "Ook tekst om te vertalen", -1},
{"", "99 bottles of beer on the wall", "99 flessen bier op de muur", 1}, {"", "99 bottles of beer on the wall", "99 flessen bier op de muur", -1},
{"", "%d apple(s)", "%d appel", 1}, {"", "%d apple(s)", "%d appel", 1},
{"", "%d apple(s)", "%d appels", 42}, {"", "%d apple(s)", "%d appels", 42},
// All of them are now in section "", lookup with original section should return the key. // All of them are now in section "", lookup with original section should return the key.
{"Page", "Text for translation", "Text for translation", 1}, {"Page", "Text for translation", "Text for translation", -1},
{"Page", "Also text to translate", "Also text to translate", 1}, {"Page", "Also text to translate", "Also text to translate", -1},
{"installscript", "99 bottles of beer on the wall", "99 bottles of beer on the wall", 1}, {"installscript", "99 bottles of beer on the wall", "99 bottles of beer on the wall", -1},
{"apple_count", "%d apple(s)", "%d apple(s)", 1}, {"apple_count", "%d apple(s)", "%d apple(s)", 1},
{"apple_count", "%d apple(s)", "%d apple(s)", 42}, {"apple_count", "%d apple(s)", "%d apple(s)", 42},
}, },
@@ -175,7 +177,7 @@ test :: proc(t: ^testing.T, suite: Test_Suite, loc := #caller_location) {
if err == .None { if err == .None {
for test in suite.tests { for test in suite.tests {
val := T(test.section, test.key, test.n, cat) val := test.n > -1 ? Tn(test.section, test.key, test.n, cat): T(test.section, test.key, cat)
testing.expectf(t, val == test.val, "Expected key `%v` from section `%v`'s form for value `%v` to equal `%v`, got `%v`", test.key, test.section, test.n, test.val, val, loc=loc) testing.expectf(t, val == test.val, "Expected key `%v` from section `%v`'s form for value `%v` to equal `%v`, got `%v`", test.key, test.section, test.n, test.val, val, loc=loc)
} }
} }