Tetum (tet) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | tetwiki sample |
|---|---|
| original | manelima (iha lia-portugés: ''manelima'') suku ida iha timór lorosa'e nian. suku ne'e iha postu administrativu laklubar (munisípiu manatutu). populasa haupu (iha lia-portugés: ''haupu'') suku ida iha timór lorosa'e nian. suku ne'e iha postu administrativu letefuó (munisípiu ermera). populasaun 0000 ( the land force of the f-fdtl consists of two light infantry battalions, each with an authorised strength of 000 personnel. the force was predominantly |
| 1000 | ▁mane lima ▁( iha ▁lia - portugés : ▁'' man e lima '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁la klu bar ▁( munisípiu ▁manatutu ). ▁popu la sa ▁ha u pu ▁( iha ▁lia - portugés : ▁'' ha u pu '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁le te f u ó ▁( munisípiu ▁ermera ). ▁populasaun ▁0000 ▁( ▁the ▁l and ▁for ce ▁of ▁the ▁f - fdtl ▁con s ist s ▁of ▁t w o ▁li ght ▁in f an tr y ▁ba t ta li on s , ▁e a ch ▁w it h ▁an ▁au th oris e d ▁s tr en g th ▁of ▁000 ▁per s on n el . ▁the ▁for ce ▁w as ▁pre d om inan tl y |
| 3000 | ▁mane lima ▁( iha ▁lia - portugés : ▁'' mane lima '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁laklu bar ▁( munisípiu ▁manatutu ). ▁popu la sa ▁hau pu ▁( iha ▁lia - portugés : ▁'' ha u pu '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁lete fu ó ▁( munisípiu ▁ermera ). ▁populasaun ▁0000 ▁( ▁the ▁l and ▁force ▁of ▁the ▁f - fdtl ▁con s ist s ▁of ▁t wo ▁li ght ▁inf an tr y ▁bat tali ons , ▁e a ch ▁with ▁an ▁au th oris ed ▁s tr eng th ▁of ▁000 ▁personnel . ▁the ▁force ▁was ▁pre dom inan tl y |
| 5000 | ▁mane lima ▁( iha ▁lia - portugés : ▁'' mane lima '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁laklubar ▁( munisípiu ▁manatutu ). ▁popu la sa ▁hau pu ▁( iha ▁lia - portugés : ▁'' hau pu '') ▁suku ▁ida ▁iha ▁timór ▁lorosa ' e ▁nian . ▁suku ▁ne ' e ▁iha ▁postu ▁administrativu ▁letefuó ▁( munisípiu ▁ermera ). ▁populasaun ▁0000 ▁( ▁the ▁land ▁force ▁of ▁the ▁f - fdtl ▁cons ist s ▁of ▁two ▁li ght ▁infan try ▁battali ons , ▁ea ch ▁with ▁an ▁auth oris ed ▁strength ▁of ▁000 ▁personnel . ▁the ▁force ▁was ▁pre dom inan tl y |