This a great website not only describing how web crawlers work, but how you can create on your own. Descriptions and pictures really help to create one if your stuck too.
That is a good tutorial -- thanks, Gordon. My own PHP skills are good enough to build this, though at the moment I don't need to. I had forgotten the synonym "scrapers," too. Useful quotation: "One typical task that Google performs is to pull all the links from a page and see which sites they are endorsing."
Yes, it is! I've already given you credit for one link this week, but that is a good explanation -- even though the page it was last updated in 2005 and the search engine it gives as an example is Alta Vista. :) Alta Vista doesn't even exist anymore; it was acquired by Yahoo awhile ago, I think. Here's the Wikipedia page: https://en.wikipedia.org/wiki/AltaVista