Who is to blame?
w
No one and everyone:
n
Directories don�t care about completeness
n
Indexes can�t keep up with growth of pages
n
Webmasters may not welcome spiders
n
We
don�t want to pay for information