Tswana (tn) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | tnwiki sample |
|---|---|
| original | hambukushu ba dumela gore modimo wa bone ebong, nyambe, o tsentse letso le leruo la bone mo thabeng ya mosadi. bosupi jo bo tletseng bo bonwa ka dikga that part of business enterprise which is devoted to the direction and co-ordination of its various activities." hierarchy e kaya peakanyo le moetaledipele leader yo o etelelang pele maloko mangwe a okanaiseišene. peakanyo e go le gontsi e goloanngwa le gore go n |
| 1000 | ▁ha m bu ku s hu ▁ba ▁du mela ▁gore ▁mo dimo ▁wa ▁bone ▁e bong , ▁nya m be , ▁o ▁tse ntse ▁le tso ▁le ▁le ru o ▁la ▁bone ▁mo ▁tha be ng ▁ya ▁mo sadi . ▁bosu pi ▁jo ▁bo ▁tle tseng ▁bo ▁bonwa ▁ka ▁dikga ▁tha t ▁pa rt ▁of ▁bu si ne ss ▁e nte r p r ise ▁w hi ch ▁is ▁de v o ted ▁to ▁the ▁dire c tion ▁and ▁co - or di na tion ▁of ▁i ts ▁v a ri o u s ▁a c ti vi ti es . " ▁hi e ra r ch y ▁e ▁ka ya ▁pe a kanyo ▁le ▁moe ta ledi pele ▁le a der ▁yo ▁o ▁e te lela ng ▁pele ▁ma loko ▁mangwe ▁a ▁oka na ise i š e ne . ▁pe a kanyo ▁e ▁go ▁le ▁go ntsi ▁e ▁go lo an ngwa ▁le ▁gore ▁go ▁n |
| 3000 | ▁ha m bu ku s hu ▁ba ▁dumela ▁gore ▁modimo ▁wa ▁bone ▁ebong , ▁nya m be , ▁o ▁tse ntse ▁letso ▁le ▁leruo ▁la ▁bone ▁mo ▁tha beng ▁ya ▁mosadi . ▁bosu pi ▁jo ▁bo ▁tle tseng ▁bo ▁bonwa ▁ka ▁dikga ▁that ▁pa rt ▁of ▁bu si ness ▁e nter p r ise ▁w hi ch ▁is ▁de vo ted ▁to ▁the ▁dire c tion ▁and ▁co - or di na tion ▁of ▁its ▁va ri o us ▁ac ti vi ties ." ▁hi e ra rch y ▁e ▁kaya ▁pe a kanyo ▁le ▁moe ta ledi pele ▁lea der ▁yo ▁o ▁ete lelang ▁pele ▁maloko ▁mangwe ▁a ▁okana iseišene . ▁pe a kanyo ▁e ▁go ▁le ▁go ntsi ▁e ▁golo an ngwa ▁le ▁gore ▁go ▁n |
| 5000 | ▁ha mbu kushu ▁ba ▁dumela ▁gore ▁modimo ▁wa ▁bone ▁ebong , ▁nya mbe , ▁o ▁tse ntse ▁letso ▁le ▁leruo ▁la ▁bone ▁mo ▁thabeng ▁ya ▁mosadi . ▁bosupi ▁jo ▁bo ▁tletseng ▁bo ▁bonwa ▁ka ▁dikga ▁that ▁pa rt ▁of ▁bu siness ▁enter p rise ▁whi ch ▁is ▁de vo ted ▁to ▁the ▁dire ction ▁and ▁co - or dina tion ▁of ▁its ▁va ri ous ▁ac ti vi ties ." ▁hi e ra rch y ▁e ▁kaya ▁pe akanyo ▁le ▁moe ta ledi pele ▁lea der ▁yo ▁o ▁ete lelang ▁pele ▁maloko ▁mangwe ▁a ▁okanaiseišene . ▁pe akanyo ▁e ▁go ▁le ▁gontsi ▁e ▁golo an ngwa ▁le ▁gore ▁go ▁n |