Inupiaq (ik) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | ikwiki sample |
|---|---|
| original | jeesus kristus kristo redentor, rio de janeiro, brazil. jeesus kristus on jumala poeg ning vanas testamendis ennustatud messias. kristlased käsitlevad * emma bodfish suli david baumgartner (0000), iñupiat grammar. utqiaġvigmi: utqiaġvium minuaqtuġviata iñupiatun savagvianni * jonathan m. nielson (000 when it comes to planning a wedding, i always advise people to start with a budget. the budget is the framework that tells you how much wedding you ca |
| 1000 | ▁jeesus ▁krist us ▁krist o ▁r ed ent or , ▁ri o ▁de ▁jane ir o , ▁b r az il . ▁jeesus ▁krist us ▁on ▁j um ala ▁p o eg ▁ning ▁van as ▁testamend is ▁en n ust atud ▁m es si as . ▁kristlased ▁käsitle v ad ▁* ▁ em ma ▁b od f ish ▁suli ▁d av id ▁b a um g art n er ▁(0000), ▁iñupiat ▁g ram m ar . ▁utqiaġv ig mi : ▁utqiaġv ium ▁m in uaq tu ġv i ata ▁iñupiatun ▁sav ag v i ann i ▁* ▁j on at h an ▁m . ▁ni el son ▁( 000 ▁wh en ▁it ▁com es ▁to ▁pl ann ing ▁a ▁wedding , ▁i ▁al way s ▁a d v is e ▁people ▁to ▁start ▁with ▁a ▁budget . ▁the ▁budget ▁is ▁the ▁f ram ew ork ▁that ▁t ell s ▁y ou ▁h ow ▁m u ch ▁wedding ▁y ou ▁c a |
| 3000 | ▁jeesus ▁kristus ▁krist o ▁r edent or , ▁rio ▁de ▁jane iro , ▁b razil . ▁jeesus ▁kristus ▁on ▁jumala ▁poeg ▁ning ▁vanas ▁testamendis ▁ennustatud ▁messias . ▁kristlased ▁käsitlevad ▁* ▁emma ▁bod fish ▁suli ▁david ▁baumg artner ▁(0000), ▁iñupiat ▁gram mar . ▁utqiaġv igmi : ▁utqiaġv ium ▁min uaqtu ġviata ▁iñupiatun ▁savag vianni ▁* ▁j onath an ▁m . ▁ni elson ▁( 000 ▁when ▁it ▁comes ▁to ▁planning ▁a ▁wedding , ▁i ▁alway s ▁ad vise ▁people ▁to ▁start ▁with ▁a ▁budget . ▁the ▁budget ▁is ▁the ▁f ramew ork ▁that ▁tells ▁you ▁how ▁much ▁wedding ▁you ▁c a |
| 5000 | ▁jeesus ▁kristus ▁kristo ▁redentor , ▁rio ▁de ▁janeiro , ▁brazil . ▁jeesus ▁kristus ▁on ▁jumala ▁poeg ▁ning ▁vanas ▁testamendis ▁ennustatud ▁messias . ▁kristlased ▁käsitlevad ▁* ▁emma ▁bodfish ▁suli ▁david ▁baumgartner ▁(0000), ▁iñupiat ▁grammar . ▁utqiaġvigmi : ▁utqiaġvium ▁minuaqtuġviata ▁iñupiatun ▁savagvianni ▁* ▁jonathan ▁m . ▁nielson ▁( 000 ▁when ▁it ▁comes ▁to ▁planning ▁a ▁wedding , ▁i ▁always ▁advise ▁people ▁to ▁start ▁with ▁a ▁budget . ▁the ▁budget ▁is ▁the ▁framework ▁that ▁tells ▁you ▁how ▁much ▁wedding ▁you ▁ca |