Is white space tokenization enough?
- Enter several sample sentences (you can copy paste them from the web or write your own) into the textbox where it says “tokenize text”. Your sentences should include at least one contraction and at least one compound word (if you don’t know what a compound word is, see here).
“I can’t climb on my bedframe.”
“I don’t care about your cheeseburger.”
2. Observe how the different tokenizers handle your text. Look carefully at the whitespace tokenizer and answer the following question: Are spaces sufficient to tokenize English language text? Why or why not? Cite examples from your text to support your conclusion.
Spaces are sometimes sufficient to tokenize English language text. In some cases, spaces broke up compound words and/or contractions. In other cases, though, spacers left contractions and compounds together. This could cause confusion in language processing.
Try out a CALL tool
- Identify one CALL tool that you could try out. Spend at least 10 minutes trying out the tool. Possible examples include, but are not limited to, Duolingo, Mango Languages, Babble, Rosetta Stone, etc.
2. What type of feedback does the tool give? Is it individualized feedback?
Duolingo provides individualized feedback by correcting every word you got wrong in a sentence.
3. How does the tool handle clozes? Does it allow multiple possibly correct answers?
Duolingo handles clozes by offering multiple choices, but only accepts one correct answer.
4. Does the tool allow you to work on, around or through the language?
Duolingo allows you to work on the language by encouraging users to learn multiple words and phrases in a language they want to learn.