BPEmb: bm

Bambara (bm) subword embeddings

Vocab size	vocab	model	25 dim	50 dim	100 dim	200 dim	300 dim
1000	vocab	model	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix
3000	vocab	model	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix
5000	vocab	model	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix	txt \| bin bokeh \| umap \| matrix

View Bambara subword embeddings in TensorFlow Projector

2-D UMAP plots

Embedding matrix plots

Training corpus sample, encoded with different BPE vocabulary sizes

Vocab size	bmwiki sample
original	;wikipedia afrika-kan wɛrɛw minu tɛ carinen baw ye afrikaans · akan · አማርኛ · chi-chewa · chishona · chitumbuka · ɛʋɛ · fulfulde · gĩkũyũ · هَوُسَ · ig jagomajira ye jagofεn kofɔli ye k'a mankutu fɔ a ka dya a magotigiw ye u ka na dɔ лini a la. jagomajira kεbaa - jagomajirala - bε lakalili kε jagofεn a dugu man bon, dugu den te tɛmɛ dugu den baa kelen kan,u'ka baara ye sɛnɛ, cago, banga balo ni môni. kalan so te ni dugula, dôgôtôrô so te ni dugula,
1000	▁ ; wi ki p edi a ▁afrika - kan ▁wɛrɛ w ▁minu ▁tɛ ▁ca rin en ▁baw ▁ye ▁afrika an s ▁· ▁a kan ▁· ▁ አማርኛ ▁· ▁c hi - c he wa ▁· ▁c hi sh ona ▁· ▁c hi t um bu ka ▁· ▁ ɛ ʋ ɛ ▁· ▁fu l f u l de ▁· ▁g ĩ k ũ y ũ ▁· ▁ ه َ و ُ س َ ▁· ▁i g ▁jago ma ji ra ▁ye ▁jago f ε n ▁ko fɔ li ▁ye ▁k ' a ▁man ku tu ▁fɔ ▁a ▁ka ▁d ya ▁a ▁ma go tigi w ▁ye ▁u ▁ka ▁na ▁dɔ ▁ л ini ▁a ▁la . ▁jago ma ji ra ▁k ε baa ▁- ▁jago ma ji ra la ▁- ▁b ε ▁laka li li ▁k ε ▁jago f ε n ▁a ▁dugu ▁man ▁bon , ▁dugu ▁den ▁te ▁tɛmɛ ▁dugu ▁den ▁ba a ▁kelen ▁kan , u ' ka ▁baara ▁ye ▁sɛnɛ , ▁ca go , ▁ban ga ▁ba lo ▁ni ▁m ô ni . ▁kalan ▁so ▁te ▁ni ▁dugu la , ▁dô gô t ôrô ▁so ▁te ▁ni ▁dugu la ,
3000	▁; wi ki pedia ▁afrika - kan ▁wɛrɛw ▁minu ▁tɛ ▁carin en ▁baw ▁ye ▁afrika an s ▁· ▁a kan ▁· ▁ አማርኛ ▁· ▁chi - che wa ▁· ▁chi sh ona ▁· ▁chi t um bu ka ▁· ▁ɛ ʋ ɛ ▁· ▁fu lf ul de ▁· ▁g ĩ k ũ y ũ ▁· ▁ ه َ و ُ س َ ▁· ▁i g ▁jagomajira ▁ye ▁jago f ε n ▁ko fɔ li ▁ye ▁k ' a ▁manku tu ▁fɔ ▁a ▁ka ▁d ya ▁a ▁mago tigiw ▁ye ▁u ▁ka ▁na ▁dɔ ▁л ini ▁a ▁la . ▁jagomajira ▁k ε baa ▁- ▁jagomajira la ▁- ▁b ε ▁lakali li ▁k ε ▁jago f ε n ▁a ▁dugu ▁man ▁bon , ▁dugu ▁den ▁te ▁tɛmɛ ▁dugu ▁den ▁baa ▁kelen ▁kan , u ' ka ▁baara ▁ye ▁sɛnɛ , ▁ca go , ▁ban ga ▁balo ▁ni ▁mô ni . ▁kalan ▁so ▁te ▁ni ▁dugula , ▁dôgô t ôrô ▁so ▁te ▁ni ▁dugula ,
5000	▁; wi ki pedia ▁afrika - kan ▁wɛrɛw ▁minu ▁tɛ ▁carin en ▁baw ▁ye ▁afrika an s ▁· ▁akan ▁· ▁ አማርኛ ▁· ▁chi - che wa ▁· ▁chi sh ona ▁· ▁chi tum bu ka ▁· ▁ɛ ʋ ɛ ▁· ▁fu lfulde ▁· ▁g ĩ k ũ y ũ ▁· ▁ ه َ و ُ س َ ▁· ▁i g ▁jagomajira ▁ye ▁jagof ε n ▁ko fɔli ▁ye ▁k ' a ▁mankutu ▁fɔ ▁a ▁ka ▁d ya ▁a ▁mago tigiw ▁ye ▁u ▁ka ▁na ▁dɔ ▁л ini ▁a ▁la . ▁jagomajira ▁k ε baa ▁- ▁jagomajira la ▁- ▁b ε ▁lakali li ▁k ε ▁jagof ε n ▁a ▁dugu ▁man ▁bon , ▁dugu ▁den ▁te ▁tɛmɛ ▁dugu ▁den ▁baa ▁kelen ▁kan , u ' ka ▁baara ▁ye ▁sɛnɛ , ▁cago , ▁banga ▁balo ▁ni ▁môni . ▁kalan ▁so ▁te ▁ni ▁dugula , ▁dôgô t ôrô ▁so ▁te ▁ni ▁dugula ,