Problem
If you are seeing 401.1 unauthorized event log entries by the SharePoint Gatherer in your event log, this article may be what you've been googling for.
Are you able to browse to your sharepoint site from the server logged in as the search crawling account? No luck? Keep reading.
Reason
With the Windows Server 2003 SP1 ( or possibly just the .Net Framework 3.5 SP1), Microsoft introduced a new security feature that checks for loopbacks (making calls to web applications locally from thet server). If your search service is running on the same server that is hosting your web applications (which is best practice for indexing speed), you'll most likely recieve this unauthorized error. You may also see this error for any custom applications which use the SharePiont web services locally. This security fix is meant to block malicious applications from infultrating your web sites from within, which is a good fix in my opinion.
How to I get around this 401 unauthorized then?
Microsoft recommends two fixes. Both of them are registry modifications.
One turns off the Loopback check altogether which shouldn't be done on external facing production servers for obvious reasons. The other is to specify the sites that allow loopback from the server.
Here's the instructions for the registry modifications required for both methods:
MS KB Article 896861
Tags: sharepoint, wss 3.0, sharepoint search, disableloopbackcheck