FORTUNE -- Dear Annie: I had a strange experience the other day: I went to a second round of job interviews at a company where I'd really like to work, and a manager there mentioned that he, too, is a fan of a little-known science fiction writer whose books I happen to like.
That was nice, since it gave us something in common besides work, and we had a pleasant conversation. But it dawned on me later that the only way he could possibly have known I liked this writer was, if he had seen my "wish list" on Amazon.com. Is that something employers usually look at? It seems weird. And if they're looking at that, what else are they looking at? -Creeped Out
Dear C.O.: Oh my. The good news is, this company seems to be seriously interested in hiring you, because they've apparently bothered to do -- or, more likely, paid someone else to do -- what's called a deep-Internet search, to glean every scrap of information about you they possibly can. The not-so-cheery news is, they might know a lot more about you than you realize.
The so-called deep Internet (also known as the Deepnet, the invisible Web, or the dark Web) is not new, but enterprising techies have recently come up with ever more sophisticated algorithms for trolling its vast contents. To get an idea of the size of the deep Web, consider: Researchers estimate it's more than 500 times the size of the everyday Internet you can see with an ordinary search engine.
Someone adept at deep-Web diving can find information in databases that have blocked traditional search engines, as well as certain kinds of multimedia files and other formats Google can't reach -- including Web pages unlinked to any other pages, data from password-protected sites, and much, much more.
In practical terms, says Lori Fenstermaker, CEO of online recruiters AutoSearch, this means that "Amazon wish lists can crop up. So can your results from the last marathon you ran, and whose political campaign you've given money to, and whether your house is in foreclosure." Ever filed an application for a patent? Declared bankruptcy? Fallen behind on your child-support payments? Been investigated by the Securities and Exchange Commission? A Google (GOOG, Fortune 500) search probably won't reveal any of that, but a deep-Web search could.
"Another thing is, every blog comment you ever posted is liable to show up," says Fenstermaker. "People really should think twice before putting up nasty posts on blogs, especially if they are long enough that you obviously put some thought into them, because employers look askance at people who seem chronically angry or who can't disagree in a civil way. It's a definite red flag."
Indeed, hiring managers often look for any negative mentions of a past or present boss or employer, says Jonathan Schreiber, senior vice president of business development at Pipl, a company that specializes in conducting deep Web searches for recruiters and Fortune 500 companies.
"If you're sending Tweets to your friends bashing the company where you work or used to work, that will turn up," he says. "We don't tell employers who to hire or not hire, we just provide the data, but many of them see that as a dealbreaker."
It's not hard for employers to have these types of searches conducted, either. Pipl, which techie blog TechCrunch.com once described as "a search engine so good, it will scare your pants off," will generate a report on a candidate's deep-Web presence for free, if an employer does just a few per day. For huge clients that may want thousands of searches on a regular basis, fees range from $7,500 to $10,000 a month.
Still, the searches only go so far. "Our searches don't intrude into anything you have designated as private," Schreiber says. "If you have set your privacy controls on Facebook to allow only friends to see your information, for example, we don't go beyond that barrier. It wouldn't be ethical." Likewise, secure sites like bank accounts are off limits, he says.
Even so, says Schreiber, "most people have put a lot more data out there, in various places, than they realize. Blog comments, for instance, seem 'anonymous' to many people because they think the Internet is so vast, how could anyone find that? But the fact is, an employer who does a Pipl search finds a lot more than just your LinkedIn profile. They're looking at a pattern of online behavior that can reveal your whole personality."
Fighting back
In your case, there seems to have been no harm done: You're apparently still in the running for the job, and you found a common interest with a possible future colleague. But, for anyone who suspects there's some dirt on them in the deep Web they'd prefer employers not to see, all is not lost. There are online services that will dig it up and expunge it, for a modest fee. They'll also monitor the deep Web, and alert you if anything new crops up. Reputation Defender, the first and biggest of these outfits, charges around $15 a month to keep your deep-Web image squeaky clean.
"Control of your digital information, your online self, is a problem that is getting bigger every day," says Michael Fertik, who started Reputation Defender in 2006. The company now has customers in 45 countries. "Our research shows that about 14% of employers now are even delving into really obscure parts of the Internet, like virtual worlds" -- evidently on the dubious assumption that your World of Warcraft avatar reveals something about you that an interviewer needs to know.
Note to hiring managers: Before you embark on a deep Web search (or hire someone else to do it), have a word with your company's attorney. Peter Gillespie, an employment lawyer at Fisher & Phillips in Chicago, discourages his corporate clients from deep Web diving. Why? "You run too big a risk of finding out something you would not be allowed to ask in an interview," he says.
For instance, hiring managers are prohibited by law from asking if an applicant has ever had cancer. What if a deep Web search reveals his or her membership in a cancer survivors' support group? "Are you going to be able to put that completely out of your mind?" says Gillespie. "What if you decide not to hire this person for some other reason, but he or she hits you with a lawsuit claiming it was an ADA (Americans with Disabilities Act) violation?"
He adds: "HR departments have policies in place that are carefully designed to stay within the law, so stick with those." If that approach seems outdated, Gillespie points out, "bear in mind that employers were somehow able to make perfectly good hiring decisions before the Internet even existed."
