Kalaallisut (kl) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | klwiki sample |
|---|---|
| original | thule kulturi ukiut 0000 kingornatigut inuit avanersuaq tikeqqaarpaa, inuit-lu issittumi ullumikkut inuusut siuleraat. qaanaaq inuusoqaaq oqaluttuaris singles * 0000 every little part of me * 0000 shout (out love will be the light) * 0000 completely fallen * 0000 it's a wonderful feeling naalagaaffiit peqatigiit inuisa isumaqatigiissutikkut uppernarsaqqimmassuk inuttut pisinnaatitaaffiit tunngaviusut, inuup ataqqinassusiata erianassusi |
| 1000 | ▁t h u le ▁k ul tu ri ▁uki ut ▁0000 ▁king orn atigut ▁inuit ▁av an ersu aq ▁t ik eqq aar paa , ▁inuit - lu ▁i ss ittu mi ▁ullumi kkut ▁inu usut ▁si uler aat . ▁q aan aaq ▁inu usoq aaq ▁oqaluttu aris ▁s ing le s ▁* ▁0000 ▁e ver y ▁l itt le ▁p art ▁of ▁m e ▁* ▁0000 ▁s ho ut ▁( o ut ▁l ov e ▁w ill ▁b e ▁the ▁ lig ht ) ▁* ▁0000 ▁c om p le t el y ▁f all en ▁* ▁0000 ▁it ' s ▁a ▁w on der f ul ▁f e el ing ▁naalag aaffi it ▁p eqatigi it ▁inu is a ▁isumaq atigi iss ut i kkut ▁upp ern ars aqq i mma ssu k ▁in uttut ▁pisinnaatit aaffi it ▁tunngav i usut , ▁inuup ▁at aqq in ass usi ata ▁ eri an ass usi |
| 3000 | ▁th ule ▁kultu ri ▁ukiut ▁0000 ▁kingorn atigut ▁inuit ▁av an ersuaq ▁tik eqqaar paa , ▁inuit - lu ▁iss ittumi ▁ullumikkut ▁inu usut ▁si uler aat . ▁qaan aaq ▁inu usoq aaq ▁oqaluttuaris ▁sing les ▁* ▁0000 ▁e ver y ▁l itt le ▁part ▁of ▁me ▁* ▁0000 ▁s ho ut ▁( o ut ▁l ov e ▁w ill ▁be ▁the ▁ lig ht ) ▁* ▁0000 ▁com ple t el y ▁f all en ▁* ▁0000 ▁it ' s ▁a ▁w on der f ul ▁fe el ing ▁naalagaaffiit ▁peqatigi it ▁inu isa ▁isumaqatigi iss uti kkut ▁uppernars aqq i mma ssuk ▁inuttut ▁pisinnaatit aaffiit ▁tunngavi usut , ▁inuup ▁ataqq in ass usi ata ▁ eri an ass usi |
| 5000 | ▁thule ▁kulturi ▁ukiut ▁0000 ▁kingorn atigut ▁inuit ▁av an ersuaq ▁tik eqqaar paa , ▁inuit - lu ▁iss ittumi ▁ullumikkut ▁inu usut ▁si uler aat . ▁qaanaaq ▁inu usoq aaq ▁oqaluttuaris ▁sing les ▁* ▁0000 ▁e ver y ▁l itt le ▁part ▁of ▁me ▁* ▁0000 ▁sho ut ▁( out ▁love ▁will ▁be ▁the ▁ lig ht ) ▁* ▁0000 ▁com ple tel y ▁f allen ▁* ▁0000 ▁it ' s ▁a ▁w on der f ul ▁fe el ing ▁naalagaaffiit ▁peqatigiit ▁inu isa ▁isumaqatigi iss uti kkut ▁uppernars aqq i mma ssuk ▁inuttut ▁pisinnaatit aaffiit ▁tunngavi usut , ▁inuup ▁ataqqin assusi ata ▁eri an assusi |