Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
5

If your authentication is working there is no reason to hide the url. Id rather focus on that.


4

It is not possible for search engines to find it, if it is not linked anywhere (as this is the only way search engines find other information). In order to make sure a search engine does not find it via a robots.txt and have the secrecy, use a double-hash: http://example.com/asdfghjk/12345678 Your robots.txt would disallow anything below asdfghjk: User-...


2

You want to throttle the bot, but you don't appear to know WHY you want to do this. Are you experiencing a performance impact? Is the traffic pushing you over a bandwidth or transfer threshold? Throttling a bot "just because" is a waste of effort - If it is not hurting you I suggest that you leave it alone. If it is causing problems you can take steps ...


2

I think you want Apache Solr. Lucene is a library, while Solr is a Tomcat application, much more likely what your friend was thinking of.


2

This error can result if there is any issue while importing haystack/backends/whoosh_backend.py. One such example is an import error on from whoosh.spelling import SpellChecker As a result of mixing Whoosh version 2.5.1 with Haystack 1.2.7 (or likely other 1.2.x version of Haystack. The recommended fix for now is to install the older 2.4.1 version of ...


1

I don't think that is provider of any services Google uses. The first result for Google search on that link gives me this page, this seems to be a malicious redirect. http://evuln.com/labs/humbertech.com/ Your site might be infected with some malware, check your .htacces and php files for any redirects. Also I think this would be more suited for https://...


1

Create robots.txt file with the following contents under (/var/www/html): User-agent: YandexBot User-agent: Ezooms Disallow: / Add the following to your httpd.conf file Alias /robots.txt /var/www/html/robots.txt This robots.txt file will now be served for all virtual hosts on your server, overriding any robots.txt file you might have for individual hosts....


1

This is a really hard one to answer, and certainly not something you'll get a clear answer from google or amazon on. We've gone through the same routes/questions and we're a major SEO player in the UK with Google. The long and short is that no-one really seems to know or can be sure, some people say yes others say no. If SEO and search results is a big ...


1

Like a Google Mini? (didn't think they still had those, but it looks that way)


1

Well, the regex in your RewriteCond demands that the User Agent start with bingbot. That's what the ^ in the regex does. ^bingbot/.* Since the User Agent (from your log example) doesn't start with that, it won't match and skips the Rule. "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" Remove the ^ and it should work, though I've ...


1

This blog post is one of the better resources I've come across. It provides a great overview, downloadable ADMX and ADML files, as well as what's required to properly implement them through a GPO.


Only top voted, non community-wiki answers of a minimum length are eligible