Fulah (ff) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | ffwiki sample |
|---|---|
| original | konaakri kapital (laamorde) leydi gine, takko baharu atalantik, yimɓe hoɗuɓe e mayre no yaha wa miliyon gooto e ko fawi (woɓɓe no tuumi miliyonji 0 (0 {| border=0 align=right cellpadding=0 cellspacing=0 width=000 style="margin: 0 0 0em 0em; background: #f0f0f0; border: 0px #aaaaaa solid; border-colla {| border=0 align=right cellpadding=0 cellspacing=0 width=000 style="margin: 0 0 0em 0em; background: #f0f0f0; border: 0px #aaaaaa solid; border-colla |
| 1000 | ▁kon aak ri ▁ka pital ▁( laam orde ) ▁leydi ▁gine , ▁ta kk o ▁b ah ar u ▁ at al an ti k , ▁yimɓe ▁hoɗ uɓe ▁e ▁may re ▁no ▁yah a ▁wa ▁m il iyon ▁gooto ▁e ▁ko ▁f aw i ▁( wo ɓɓe ▁no ▁tuu mi ▁m il iyon ji ▁0 ▁( 0 ▁{| ▁border =0 ▁align = right ▁cellpadding =0 ▁cellspacing =0 ▁width =000 ▁style =" margin : ▁0 ▁0 ▁0 em ▁0 em ; ▁background : ▁# f 0 f 0 f 0; ▁border : ▁0 px ▁# aaaaaa ▁solid ; ▁border - c olla ▁{| ▁border =0 ▁align = right ▁cellpadding =0 ▁cellspacing =0 ▁width =000 ▁style =" margin : ▁0 ▁0 ▁0 em ▁0 em ; ▁background : ▁# f 0 f 0 f 0; ▁border : ▁0 px ▁# aaaaaa ▁solid ; ▁border - c olla |
| 3000 | ▁kon aak ri ▁ka pital ▁( laam orde ) ▁leydi ▁gine , ▁takko ▁bah aru ▁at al an tik , ▁yimɓe ▁hoɗ uɓe ▁e ▁mayre ▁no ▁yah a ▁wa ▁miliyon ▁gooto ▁e ▁ko ▁fawi ▁( wo ɓɓe ▁no ▁tuu mi ▁miliyonji ▁0 ▁( 0 ▁{| ▁border =0 ▁align = right ▁cellpadding =0 ▁cellspacing =0 ▁width =000 ▁style =" margin : ▁0 ▁0 ▁0 em ▁0 em ; ▁background : ▁# f 0 f 0 f 0; ▁border : ▁0 px ▁# aaaaaa ▁solid ; ▁border - c olla ▁{| ▁border =0 ▁align = right ▁cellpadding =0 ▁cellspacing =0 ▁width =000 ▁style =" margin : ▁0 ▁0 ▁0 em ▁0 em ; ▁background : ▁# f 0 f 0 f 0; ▁border : ▁0 px ▁# aaaaaa ▁solid ; ▁border - c olla |
| 5000 | ▁kon aak ri ▁kapital ▁( laamorde ) ▁leydi ▁gine , ▁takko ▁baharu ▁atalantik , ▁yimɓe ▁hoɗuɓe ▁e ▁mayre ▁no ▁yah a ▁wa ▁miliyon ▁gooto ▁e ▁ko ▁fawi ▁( woɓɓe ▁no ▁tuu mi ▁miliyonji ▁0 ▁( 0 ▁{| ▁border =0 ▁align = right ▁cellpadding =0 ▁cellspacing =0 ▁width =000 ▁style =" margin : ▁0 ▁0 ▁0 em ▁0 em ; ▁background : ▁# f 0 f 0 f 0; ▁border : ▁0 px ▁# aaaaaa ▁solid ; ▁border - c olla ▁{| ▁border =0 ▁align = right ▁cellpadding =0 ▁cellspacing =0 ▁width =000 ▁style =" margin : ▁0 ▁0 ▁0 em ▁0 em ; ▁background : ▁# f 0 f 0 f 0; ▁border : ▁0 px ▁# aaaaaa ▁solid ; ▁border - c olla |