Google webmaster FAQs suggest that this will exclude all xml files from search:
User-agent: Googlebot
Disallow: /*.xml$
Is this legal for other bots as well?
User-agent: *
Disallow: /*.xml$
| |||
|
feedback
|
|
Yes, this is valid. Whether other bots actually understand and obey is completely up to them, though! | |||
|
feedback
|
