> Ok. I see something suspicious here. The for loop:
>
> ##
> for l in xx:
> train_tokens.append(l)
> ##
>
> assumes that we get tokens from the 'xx' token. Is this true? Are you
> sure you don't have to specifically say:
>
> ##
> for l in xx['SUBTOKENS']:
> ...
> ##
On Mon, 14 Nov 2005, enas khalil wrote:
> hello all
[program cut]
Hi Enas,
You may want to try talking with NTLK folks about this, as what you're
dealing with is a specialized subject. Also, have you gone through the
tokenization tutorial in:
http://nltk.sourceforge.net/tutorial/tok
