Zulu (zu) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | zuwiki sample |
|---|---|
| original | gne, an encyclopedia which was not a wiki, also created in january 0000, co-existed with nupedia and wikipedia early in its history; however, it has b ngokusekelwa ngujongintaba, umandela waqala ukufundela izifundo zeziqu ze-bachelor of arts (ba) euniversity of fort hare, okwakuyisikhungo esiphezulu the 'glottal stop', a catch in the breath. for some people, found in ''button'' ˈbʌʔn̩, or between vowels across words: ''deus ex machina'' ˌdeɪəsˌʔɛk |
| 1000 | ▁g ne , ▁an ▁encyclo pedia ▁which ▁was ▁not ▁a ▁wiki , ▁al so ▁c rea ted ▁in ▁j an u ar y ▁0000, ▁co - e xi s ted ▁with ▁n u pedia ▁and ▁wikipedia ▁e ar ly ▁in ▁its ▁hi st or y ; ▁ho we ver , ▁it ▁has ▁b ▁ngoku seke lwa ▁ngu jo ng in taba , ▁umandela ▁wa qala ▁uku fun de la ▁izi fundo ▁ze zi qu ▁ze - ba ch e lo r ▁of ▁ar ts ▁( ba ) ▁e uni versi ty ▁of ▁for t ▁ha re , ▁okwa ku yisi khu ngo ▁esi phezulu ▁the ▁' g lo t ta l ▁s to p ' , ▁a ▁ca t ch ▁in ▁the ▁b rea th . ▁for ▁s ome ▁pe o p le , ▁fo und ▁in ▁'' bu t t on '' ▁ ˈ b ʌʔ n ̩ , ▁or ▁be t we en ▁ vo we l s ▁ac ro ss ▁wor ds : ▁'' de us ▁ex ▁ma ch ina '' ▁ ˌ de ɪ ə s ˌʔɛ k |
| 3000 | ▁g ne , ▁an ▁encyclopedia ▁which ▁was ▁not ▁a ▁wiki , ▁also ▁crea ted ▁in ▁january ▁0000, ▁co - e xis ted ▁with ▁nupedia ▁and ▁wikipedia ▁ear ly ▁in ▁its ▁hist ory ; ▁ho we ver , ▁it ▁has ▁b ▁ngoku seke lwa ▁ngu jo ng in taba , ▁umandela ▁waqala ▁uku fun dela ▁izi fundo ▁zezi qu ▁ze - ba che lor ▁of ▁ar ts ▁( ba ) ▁e uni versity ▁of ▁for t ▁ha re , ▁okwaku yisi khungo ▁esi phezulu ▁the ▁' g lo t tal ▁s top ' , ▁a ▁ca tch ▁in ▁the ▁b rea th . ▁for ▁some ▁people , ▁found ▁in ▁'' bu t ton '' ▁ ˈ b ʌʔ n ̩ , ▁or ▁between ▁vo we ls ▁ac ross ▁wor ds : ▁'' de us ▁ex ▁ma ch ina '' ▁ ˌ de ɪ ə s ˌʔɛ k |
| 5000 | ▁g ne , ▁an ▁encyclopedia ▁which ▁was ▁not ▁a ▁wiki , ▁also ▁created ▁in ▁january ▁0000, ▁co - e xis ted ▁with ▁nupedia ▁and ▁wikipedia ▁early ▁in ▁its ▁history ; ▁however , ▁it ▁has ▁b ▁ngoku sekelwa ▁ngu jo ng intaba , ▁umandela ▁waqala ▁uku fundela ▁izifundo ▁zezi qu ▁ze - ba che lor ▁of ▁ar ts ▁( ba ) ▁e university ▁of ▁for t ▁ha re , ▁okwaku yisi khungo ▁esiphezulu ▁the ▁' g lot tal ▁s top ' , ▁a ▁ca tch ▁in ▁the ▁b rea th . ▁for ▁some ▁people , ▁found ▁in ▁'' bu t ton '' ▁ ˈ b ʌʔ n ̩ , ▁or ▁between ▁vo we ls ▁ac ross ▁words : ▁'' de us ▁ex ▁ma ch ina '' ▁ ˌ de ɪ ə s ˌʔɛ k |