English (en) subword embeddings
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | enwiki sample |
|---|---|
| original | the diets of the wealthy were rich in sugars, which promoted periodontal disease. despite the flattering physiques portrayed on tomb walls, the overwe the first project proposed and worked on was a period film to be called ''dora-heita'', but when this was deemed too expensive, attention shifted to ' neutralization with a base weaker than the acid results in a weakly acidic salt. an example is the weakly acidic ammonium chloride, which is produced |
| 1000 | ▁the ▁d iet s ▁of ▁the ▁we al th y ▁were ▁r ich ▁in ▁su g ars , ▁which ▁p rom ot ed ▁per i od on t al ▁dis e ase . ▁des p ite ▁the ▁fl at ter ing ▁ph ys i qu es ▁port ra y ed ▁on ▁to m b ▁w all s , ▁the ▁over we ▁the ▁first ▁pro ject ▁pro pos ed ▁and ▁work ed ▁on ▁was ▁a ▁per i od ▁film ▁to ▁be ▁called ▁'' d or a - he it a '', ▁but ▁when ▁this ▁was ▁de em ed ▁to o ▁exp ens ive , ▁att ent ion ▁sh if ted ▁to ▁' ▁ne ut ral iz ation ▁with ▁a ▁b ase ▁we ak er ▁than ▁the ▁ac id ▁res ult s ▁in ▁a ▁we ak ly ▁ac id ic ▁s al t . ▁an ▁ex amp le ▁is ▁the ▁we ak ly ▁ac id ic ▁am m on ium ▁ch l or ide , ▁which ▁is ▁prod uced |
| 3000 | ▁the ▁d iet s ▁of ▁the ▁we alth y ▁were ▁rich ▁in ▁sug ars , ▁which ▁promot ed ▁period on tal ▁dis e ase . ▁despite ▁the ▁fl at ter ing ▁phys i qu es ▁port ray ed ▁on ▁tom b ▁wall s , ▁the ▁over we ▁the ▁first ▁project ▁pro posed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' d or a - he ita '', ▁but ▁when ▁this ▁was ▁de em ed ▁too ▁exp ensive , ▁att ention ▁sh if ted ▁to ▁' ▁ne ut ral ization ▁with ▁a ▁base ▁we aker ▁than ▁the ▁ac id ▁result s ▁in ▁a ▁we ak ly ▁ac id ic ▁sal t . ▁an ▁example ▁is ▁the ▁we ak ly ▁ac id ic ▁am mon ium ▁ch lor ide , ▁which ▁is ▁produced |
| 5000 | ▁the ▁d iet s ▁of ▁the ▁we alth y ▁were ▁rich ▁in ▁sug ars , ▁which ▁promot ed ▁period on tal ▁disease . ▁despite ▁the ▁flat ter ing ▁phys iques ▁portray ed ▁on ▁tom b ▁wall s , ▁the ▁over we ▁the ▁first ▁project ▁proposed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' d ora - he ita '', ▁but ▁when ▁this ▁was ▁de em ed ▁too ▁exp ensive , ▁attention ▁sh if ted ▁to ▁' ▁ne ut ral ization ▁with ▁a ▁base ▁we aker ▁than ▁the ▁acid ▁results ▁in ▁a ▁we ak ly ▁acid ic ▁sal t . ▁an ▁example ▁is ▁the ▁we ak ly ▁acid ic ▁am mon ium ▁ch lor ide , ▁which ▁is ▁produced |
| 10000 | ▁the ▁diet s ▁of ▁the ▁wealthy ▁were ▁rich ▁in ▁sug ars , ▁which ▁promoted ▁period on tal ▁disease . ▁despite ▁the ▁flat tering ▁phys iques ▁portrayed ▁on ▁tomb ▁walls , ▁the ▁over we ▁the ▁first ▁project ▁proposed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' d ora - he ita '', ▁but ▁when ▁this ▁was ▁deemed ▁too ▁expensive , ▁attention ▁shif ted ▁to ▁' ▁neutral ization ▁with ▁a ▁base ▁we aker ▁than ▁the ▁acid ▁results ▁in ▁a ▁weak ly ▁acid ic ▁salt . ▁an ▁example ▁is ▁the ▁weak ly ▁acid ic ▁am mon ium ▁chlor ide , ▁which ▁is ▁produced |
| 25000 | ▁the ▁diet s ▁of ▁the ▁wealthy ▁were ▁rich ▁in ▁sug ars , ▁which ▁promoted ▁period on tal ▁disease . ▁despite ▁the ▁flat tering ▁phys iques ▁portrayed ▁on ▁tomb ▁walls , ▁the ▁over we ▁the ▁first ▁project ▁proposed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' d ora - he ita '', ▁but ▁when ▁this ▁was ▁deemed ▁too ▁expensive , ▁attention ▁shifted ▁to ▁' ▁neutral ization ▁with ▁a ▁base ▁weaker ▁than ▁the ▁acid ▁results ▁in ▁a ▁weak ly ▁acid ic ▁salt . ▁an ▁example ▁is ▁the ▁weak ly ▁acid ic ▁ammon ium ▁chloride , ▁which ▁is ▁produced |
| 50000 | ▁the ▁diets ▁of ▁the ▁wealthy ▁were ▁rich ▁in ▁sugars , ▁which ▁promoted ▁period on tal ▁disease . ▁despite ▁the ▁flat tering ▁phys iques ▁portrayed ▁on ▁tomb ▁walls , ▁the ▁over we ▁the ▁first ▁project ▁proposed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' d ora - he ita '', ▁but ▁when ▁this ▁was ▁deemed ▁too ▁expensive , ▁attention ▁shifted ▁to ▁' ▁neutral ization ▁with ▁a ▁base ▁weaker ▁than ▁the ▁acid ▁results ▁in ▁a ▁weakly ▁acidic ▁salt . ▁an ▁example ▁is ▁the ▁weakly ▁acidic ▁ammonium ▁chloride , ▁which ▁is ▁produced |
| 100000 | ▁the ▁diets ▁of ▁the ▁wealthy ▁were ▁rich ▁in ▁sugars , ▁which ▁promoted ▁periodontal ▁disease . ▁despite ▁the ▁flattering ▁phys iques ▁portrayed ▁on ▁tomb ▁walls , ▁the ▁over we ▁the ▁first ▁project ▁proposed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' dora - he ita '', ▁but ▁when ▁this ▁was ▁deemed ▁too ▁expensive , ▁attention ▁shifted ▁to ▁' ▁neutralization ▁with ▁a ▁base ▁weaker ▁than ▁the ▁acid ▁results ▁in ▁a ▁weakly ▁acidic ▁salt . ▁an ▁example ▁is ▁the ▁weakly ▁acidic ▁ammonium ▁chloride , ▁which ▁is ▁produced |