Novial (nov) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | novwiki sample |
|---|---|
| original | mangaung es un ek li komunies de moteo distrikte de liberi state provinse de sud afrika. li majoritate de lun 000 000 homes parla sutum (0000 nombro). kamiesberg es un ek li komunies de namakwa distrikte de nord kabe provinse de sud afrika. li majoritate de lun 00 000 homes parla afrikansum (0000 nom on pove skripte disu talim ke ye unsmi vido lu sembla es non-korekti, e forsan perversi even si korecti: |
| 1000 | ▁m ang a un g ▁es ▁un ▁ek ▁li ▁komunies ▁de ▁mo te o ▁distrikte ▁de ▁liberi ▁state ▁provinse ▁de ▁sud ▁afrika . ▁li ▁majoritate ▁de ▁lun ▁000 ▁000 ▁homes ▁parla ▁sutum ▁(0000 ▁nombro ). ▁ka mi es ber g ▁es ▁un ▁ek ▁li ▁komunies ▁de ▁na ma k wa ▁distrikte ▁de ▁nord ▁kabe ▁provinse ▁de ▁sud ▁afrika . ▁li ▁majoritate ▁de ▁lun ▁00 ▁000 ▁homes ▁parla ▁afrikansum ▁(0000 ▁nom ▁on ▁pove ▁skripte ▁dis u ▁ta lim ▁ke ▁ye ▁un s mi ▁vi do ▁lu ▁s emb la ▁es ▁non - k or ek ti , ▁e ▁for s an ▁per versi ▁ev en ▁si ▁k ore c ti : |
| 3000 | ▁m ang a ung ▁es ▁un ▁ek ▁li ▁komunies ▁de ▁mo te o ▁distrikte ▁de ▁liberi ▁state ▁provinse ▁de ▁sud ▁afrika . ▁li ▁majoritate ▁de ▁lun ▁000 ▁000 ▁homes ▁parla ▁sutum ▁(0000 ▁nombro ). ▁ka mi es berg ▁es ▁un ▁ek ▁li ▁komunies ▁de ▁namakwa ▁distrikte ▁de ▁nord ▁kabe ▁provinse ▁de ▁sud ▁afrika . ▁li ▁majoritate ▁de ▁lun ▁00 ▁000 ▁homes ▁parla ▁afrikansum ▁(0000 ▁nom ▁on ▁pove ▁skripte ▁disu ▁talim ▁ke ▁ye ▁un smi ▁vi do ▁lu ▁semb la ▁es ▁non - k or ek ti , ▁e ▁for san ▁per versi ▁even ▁si ▁kore c ti : |
| 5000 | ▁m ang a ung ▁es ▁un ▁ek ▁li ▁komunies ▁de ▁moteo ▁distrikte ▁de ▁liberi ▁state ▁provinse ▁de ▁sud ▁afrika . ▁li ▁majoritate ▁de ▁lun ▁000 ▁000 ▁homes ▁parla ▁sutum ▁(0000 ▁nombro ). ▁ka mi es berg ▁es ▁un ▁ek ▁li ▁komunies ▁de ▁namakwa ▁distrikte ▁de ▁nord ▁kabe ▁provinse ▁de ▁sud ▁afrika . ▁li ▁majoritate ▁de ▁lun ▁00 ▁000 ▁homes ▁parla ▁afrikansum ▁(0000 ▁nom ▁on ▁pove ▁skripte ▁disu ▁talim ▁ke ▁ye ▁un smi ▁vi do ▁lu ▁semb la ▁es ▁non - kor ekti , ▁e ▁forsan ▁per versi ▁even ▁si ▁kore c ti : |