Sundanese (su) subword embeddings
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | suwiki sample |
|---|---|
| original | elemental sulfur can be found near hot springs and volcanic regions in many parts of the world, especially along the pacific ring of fire. these occur kalapa (''cocos nucifera'') mangrupa anggota famili arecaceae (kulawarga palem), hiji-hijina spésiés dina genus ''cocos'', nu mangrupa palem badag, ja in fact, the wiener process is the only time-homogeneous stochastic process with independent increments and which is continuous in probability. these |
| 1000 | ▁el em ent al ▁s ul f ur ▁can ▁be ▁f o und ▁n e ar ▁h ot ▁sp r ing s ▁and ▁v ol c an ic ▁r eg ions ▁in ▁man y ▁part s ▁of ▁the ▁w or l d , ▁ es p ec ial ly ▁al ong ▁the ▁p ac if ic ▁r ing ▁of ▁f i re . ▁the se ▁o c c ur ▁kal ap a ▁('' c oc os ▁nu c if er a '') ▁mangrupa ▁ angg ota ▁f am il i ▁are c ac e a e ▁( k ula w arg a ▁pal em ), ▁hiji - h i j ina ▁sp é si és ▁dina ▁g en us ▁'' c oc os '', ▁nu ▁mangrupa ▁pal em ▁badag , ▁j a ▁in ▁f act , ▁the ▁w i ener ▁pro c ess ▁is ▁the ▁on ly ▁tim e - h om og en e ous ▁st o ch ast ic ▁pro c ess ▁with ▁ind ep end ent ▁in c re ment s ▁and ▁which ▁is ▁cont in u ous ▁in ▁pro b ab il ity . ▁the se |
| 3000 | ▁el ement al ▁sul f ur ▁can ▁be ▁found ▁n ear ▁h ot ▁sp r ings ▁and ▁vol c an ic ▁reg ions ▁in ▁many ▁part s ▁of ▁the ▁world , ▁es pec ially ▁al ong ▁the ▁p ac ific ▁r ing ▁of ▁f ire . ▁these ▁occur ▁kal apa ▁('' c oc os ▁nu c if era '') ▁mangrupa ▁anggota ▁fam ili ▁are c ace ae ▁( k ulaw arga ▁pal em ), ▁hiji - hi j ina ▁spésiés ▁dina ▁gen us ▁'' c oc os '', ▁nu ▁mangrupa ▁pal em ▁badag , ▁ja ▁in ▁fact , ▁the ▁w i ener ▁process ▁is ▁the ▁only ▁time - h om og ene ous ▁st o ch ast ic ▁process ▁with ▁ind ep end ent ▁inc re ment s ▁and ▁which ▁is ▁contin u ous ▁in ▁probability . ▁these |
| 5000 | ▁el ement al ▁sulf ur ▁can ▁be ▁found ▁near ▁hot ▁sp r ings ▁and ▁vol can ic ▁reg ions ▁in ▁many ▁parts ▁of ▁the ▁world , ▁especially ▁along ▁the ▁pac ific ▁r ing ▁of ▁fire . ▁these ▁occur ▁kalapa ▁('' c oc os ▁nu c if era '') ▁mangrupa ▁anggota ▁fam ili ▁are c ace ae ▁( k ulaw arga ▁pal em ), ▁hiji - hij ina ▁spésiés ▁dina ▁genus ▁'' c oc os '', ▁nu ▁mangrupa ▁pal em ▁badag , ▁ja ▁in ▁fact , ▁the ▁w i ener ▁process ▁is ▁the ▁only ▁time - h om og ene ous ▁st och ast ic ▁process ▁with ▁independ ent ▁incre ments ▁and ▁which ▁is ▁continu ous ▁in ▁probability . ▁these |
| 10000 | ▁element al ▁sulfur ▁can ▁be ▁found ▁near ▁hot ▁sp r ings ▁and ▁vol can ic ▁regions ▁in ▁many ▁parts ▁of ▁the ▁world , ▁especially ▁along ▁the ▁pacific ▁ring ▁of ▁fire . ▁these ▁occur ▁kalapa ▁('' c oc os ▁nu c if era '') ▁mangrupa ▁anggota ▁famili ▁are c aceae ▁( k ulaw arga ▁pal em ), ▁hiji - hijina ▁spésiés ▁dina ▁genus ▁'' c oc os '', ▁nu ▁mangrupa ▁pal em ▁badag , ▁ja ▁in ▁fact , ▁the ▁wi ener ▁process ▁is ▁the ▁only ▁time - h om og ene ous ▁st och astic ▁process ▁with ▁independent ▁incre ments ▁and ▁which ▁is ▁continuous ▁in ▁probability . ▁these |
| 25000 | ▁element al ▁sulfur ▁can ▁be ▁found ▁near ▁hot ▁springs ▁and ▁volcanic ▁regions ▁in ▁many ▁parts ▁of ▁the ▁world , ▁especially ▁along ▁the ▁pacific ▁ring ▁of ▁fire . ▁these ▁occur ▁kalapa ▁('' coc os ▁nu c ifera '') ▁mangrupa ▁anggota ▁famili ▁are c aceae ▁( kulawarga ▁palem ), ▁hiji - hijina ▁spésiés ▁dina ▁genus ▁'' coc os '', ▁nu ▁mangrupa ▁palem ▁badag , ▁ja ▁in ▁fact , ▁the ▁wi ener ▁process ▁is ▁the ▁only ▁time - h om og ene ous ▁stochastic ▁process ▁with ▁independent ▁incre ments ▁and ▁which ▁is ▁continuous ▁in ▁probability . ▁these |
| 50000 | ▁elemental ▁sulfur ▁can ▁be ▁found ▁near ▁hot ▁springs ▁and ▁volcanic ▁regions ▁in ▁many ▁parts ▁of ▁the ▁world , ▁especially ▁along ▁the ▁pacific ▁ring ▁of ▁fire . ▁these ▁occur ▁kalapa ▁('' cocos ▁nucifera '') ▁mangrupa ▁anggota ▁famili ▁are c aceae ▁( kulawarga ▁palem ), ▁hiji - hijina ▁spésiés ▁dina ▁genus ▁'' cocos '', ▁nu ▁mangrupa ▁palem ▁badag , ▁ja ▁in ▁fact , ▁the ▁wiener ▁process ▁is ▁the ▁only ▁time - hom ogeneous ▁stochastic ▁process ▁with ▁independent ▁increments ▁and ▁which ▁is ▁continuous ▁in ▁probability . ▁these |
| 100000 | ▁elemental ▁sulfur ▁can ▁be ▁found ▁near ▁hot ▁springs ▁and ▁volcanic ▁regions ▁in ▁many ▁parts ▁of ▁the ▁world , ▁especially ▁along ▁the ▁pacific ▁ring ▁of ▁fire . ▁these ▁occur ▁kalapa ▁('' cocos ▁nucifera '') ▁mangrupa ▁anggota ▁famili ▁arecaceae ▁( kulawarga ▁palem ), ▁hiji - hijina ▁spésiés ▁dina ▁genus ▁'' cocos '', ▁nu ▁mangrupa ▁palem ▁badag , ▁ja ▁in ▁fact , ▁the ▁wiener ▁process ▁is ▁the ▁only ▁time - homogeneous ▁stochastic ▁process ▁with ▁independent ▁increments ▁and ▁which ▁is ▁continuous ▁in ▁probability . ▁these |