Can I do the same as robots.txt through IIS settings? Telling
User-agent: * Disallow: /
in host header or through web.config?
No this is not a specified way. Either have the
There are only these two options available.
You could use a single robots.txt file stored somewhere on your web server, and redirect all requests to any site hosted on your IIS server for robots.txt to that file. That would be a change per site in IIS, but you may be able to do it programmatically with PowerShell.