Tetum (tet) subword embeddings

Vocab size vocab model 25 dim 50 dim 100 dim 200 dim 300 dim
1000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
3000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
5000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix

Embedding matrix plots

Training corpus sample, encoded with different BPE vocabulary sizes

Vocab sizetetwiki sample
original manelima (iha lia-portugés: ''manelima'') suku ida iha timór lorosa'e nian. suku ne'e iha postu administrativu laklubar (munisípiu manatutu). populasa
haupu (iha lia-portugés: ''haupu'') suku ida iha timór lorosa'e nian. suku ne'e iha postu administrativu letefuó (munisípiu ermera). populasaun 0000 (
the land force of the f-fdtl consists of two light infantry battalions, each with an authorised strength of 000 personnel. the force was predominantly
1000 ▁mane lima ▁( iha ▁lia - portugés : ▁'' man e lima '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁la klu bar ▁( munisípiu ▁manatutu ). ▁popu la sa
▁ha u pu ▁( iha ▁lia - portugés : ▁'' ha u pu '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁le te f u ó ▁( munisípiu ▁ermera ). ▁populasaun ▁0000 ▁(
▁the ▁l and ▁for ce ▁of ▁the ▁f - fdtl ▁con s ist s ▁of ▁t w o ▁li ght ▁in f an tr y ▁ba t ta li on s , ▁e a ch ▁w it h ▁an ▁au th oris e d ▁s tr en g th ▁of ▁000 ▁per s on n el . ▁the ▁for ce ▁w as ▁pre d om inan tl y
3000 ▁mane lima ▁( iha ▁lia - portugés : ▁'' mane lima '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁laklu bar ▁( munisípiu ▁manatutu ). ▁popu la sa
▁hau pu ▁( iha ▁lia - portugés : ▁'' ha u pu '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁lete fu ó ▁( munisípiu ▁ermera ). ▁populasaun ▁0000 ▁(
▁the ▁l and ▁force ▁of ▁the ▁f - fdtl ▁con s ist s ▁of ▁t wo ▁li ght ▁inf an tr y ▁bat tali ons , ▁e a ch ▁with ▁an ▁au th oris ed ▁s tr eng th ▁of ▁000 ▁personnel . ▁the ▁force ▁was ▁pre dom inan tl y
5000 ▁mane lima ▁( iha ▁lia - portugés : ▁'' mane lima '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁laklubar ▁( munisípiu ▁manatutu ). ▁popu la sa
▁hau pu ▁( iha ▁lia - portugés : ▁'' hau pu '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁letefuó ▁( munisípiu ▁ermera ). ▁populasaun ▁0000 ▁(
▁the ▁land ▁force ▁of ▁the ▁f - fdtl ▁cons ist s ▁of ▁two ▁li ght ▁infan try ▁battali ons , ▁ea ch ▁with ▁an ▁auth oris ed ▁strength ▁of ▁000 ▁personnel . ▁the ▁force ▁was ▁pre dom inan tl y