Venda (ve) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | vewiki sample |
|---|---|
| original | phalaphala fm its a radio that broadcast in tshivenda language, which is predominatly spoken in venda and the radio has dj's like jimmy, also known as :tata wa hina la nge matilweni, :vito ra wena a ri hlawuleke; :a ku te ku fuma ka wena; :ku rhandza ka wena a ku endliwe misaveni, :tanihi loko ku end murundu ndi ngoma i no tshinwa nga vha mbehu ya tshinnani. ngoma iyi i na madzina o fhambananaho zwi tshi ya nga u fhambana ha vhupo. kha manwe masia |
| 1000 | ▁pha la pha la ▁f m ▁i ts ▁a ▁ra di o ▁tha t ▁b ro a d ca st ▁in ▁tshivenda ▁la ngu a ge , ▁ w hi ch ▁i s ▁p re do mi na t l y ▁s po k en ▁in ▁venda ▁ and ▁the ▁ra di o ▁ha s ▁d j ' s ▁li ke ▁j i mm y , ▁a l so ▁k no wn ▁a s ▁: ta ta ▁wa ▁hi na ▁la ▁nge ▁ma ti l we ni , ▁: vi to ▁ra ▁we na ▁a ▁ri ▁h la w u le ke ; ▁: a ▁ku ▁te ▁ku ▁fu ma ▁ka ▁we na ; ▁: ku ▁r han dza ▁ka ▁we na ▁a ▁ku ▁ en d li we ▁mi sa ve ni , ▁: t ani hi ▁lo ko ▁ku ▁ en d ▁murundu ▁ndi ▁ngoma ▁i ▁no ▁tshinwa ▁nga ▁vha ▁mbe hu ▁ya ▁tshi n n ani . ▁ngoma ▁iyi ▁i ▁na ▁ma dzi na ▁o ▁fhambananaho ▁zwi ▁tshi ▁ya ▁nga ▁u ▁fhambana ▁ha ▁vhupo . ▁kha ▁manwe ▁ma si a |
| 3000 | ▁phala phala ▁f m ▁its ▁a ▁ra dio ▁tha t ▁b ro ad ca st ▁in ▁tshivenda ▁language , ▁w hi ch ▁is ▁pre do mi na t ly ▁s po ken ▁in ▁venda ▁and ▁the ▁ra dio ▁ha s ▁d j ' s ▁li ke ▁j imm y , ▁also ▁known ▁as ▁: ta ta ▁wa ▁hina ▁la ▁nge ▁ma ti lweni , ▁: vi to ▁ra ▁wena ▁a ▁ri ▁h la wu le ke ; ▁: a ▁ku ▁te ▁ku ▁fu ma ▁ka ▁wena ; ▁: ku ▁r han dza ▁ka ▁wena ▁a ▁ku ▁end li we ▁mi sa ve ni , ▁: tani hi ▁loko ▁ku ▁end ▁murundu ▁ndi ▁ngoma ▁i ▁no ▁tshinwa ▁nga ▁vha ▁mbe hu ▁ya ▁tshin nani . ▁ngoma ▁iyi ▁i ▁na ▁madzina ▁o ▁fhambananaho ▁zwi ▁tshi ▁ya ▁nga ▁u ▁fhambana ▁ha ▁vhupo . ▁kha ▁manwe ▁masi a |
| 5000 | ▁phalaphala ▁f m ▁its ▁a ▁radio ▁tha t ▁b ro adcast ▁in ▁tshivenda ▁language , ▁w hi ch ▁is ▁pre do mi na tly ▁s po ken ▁in ▁venda ▁and ▁the ▁radio ▁has ▁d j ' s ▁li ke ▁jimmy , ▁also ▁known ▁as ▁: ta ta ▁wa ▁hina ▁la ▁nge ▁ma tilweni , ▁: vi to ▁ra ▁wena ▁a ▁ri ▁hlawu le ke ; ▁: a ▁ku ▁te ▁ku ▁fu ma ▁ka ▁wena ; ▁: ku ▁rhan dza ▁ka ▁wena ▁a ▁ku ▁end liwe ▁mi sa ve ni , ▁: tanihi ▁loko ▁ku ▁end ▁murundu ▁ndi ▁ngoma ▁i ▁no ▁tshinwa ▁nga ▁vha ▁mbehu ▁ya ▁tshin nani . ▁ngoma ▁iyi ▁i ▁na ▁madzina ▁o ▁fhambananaho ▁zwi ▁tshi ▁ya ▁nga ▁u ▁fhambana ▁ha ▁vhupo . ▁kha ▁manwe ▁masi a |