Hi Martin,
I've got a system in place which does what you are describing for HTML
pages. some figures for you...

Number of pages: 1500
Number of records in wordlist table: 398000
Number of distinct words in table: 39000

So for my sample data you are talking about ~400 unique words per
document.

Note that my wordlist table stores the uppercase and the original case
word to allow case or case-insensitve queries.

I'll look into it and see if I can extract the relevent meta data.

Regards
Ian Newby