Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We have a website idea that is related with some specific files. What we will do is to crawl all the web and fetch some specific files, and as you guess, host them in CDNs. Some of the files could be copyrighted materials as there are a lot of open websites that host copyrighted materials. We may have 40-50 thousands copyrighted materials out of 1 million files. If we get any complaints, we'll remove the material in one business day.

However, do you think in the meantime CDNs may delete all of our files? As you may know, if you host copyrighted materials, a lot of companies make their complaints directly to the datacenters which host these files to create additional pressure on the owner. And sometimes datacenters shut down the servers without giving any notice. We will be a fully legitimate business but it is practically impossible to detect copyrighted materials.

This project is still an idea, however we would like to foresee the potential problems. What do you think? Is it just better off to create our own storage solution?

edit: I guess I could not make it clear. It is %100 same as google or yahoo or bing. We just will be a focused search engine. Google has already indexed those type of documents. So there is no problem with that. However, my question was, google is removing those type of documents if they get complaints, but they have no efforts or whatsoever to prevent indexing the documents, because it is practically impossible for google too. The main point is, google is a giant so they have no problems with that. However, small companies may have troubles. If we work with local datacenters in Europe, we will have zero problems, as we already have excellent relationships with them (we rent hundreds of servers time to time for different projects) However, we do not have any relationship with any CDNs as we did not attempt to have a global project. Anyway, I guess it is more clear now.


share|improve this question
If you are pulling stuff off other people's website i would imagine you are probably going to see 100% of your files being under one form of copyright and/or license or another. – Zypher Dec 2 '09 at 4:55
up vote 2 down vote accepted

Yuri, this seems like the most relevant information I can supply:

Basically, if you're hosted out of the US, you can structure your system so that you're protected under the DMCA safe harbour provisions.

Depending on the hosting provider you go with, some will flip out and cancel small accounts if they get complaints about copyright materials. You can mitigate this by going with a larger provider and talking with them beforehand about handling DMCA requests.

You'll also need to provide a clear 'report copyright content' path for visitors to report DMCA violations to you. This will reduce the number of users jumping up the chain and talking to your provider.

share|improve this answer
Thanks, yes that DMCA information will be the first thing we will provide in the website. Thanks again for the link and the information.. – Yuri Dec 2 '09 at 19:24

We will be a fully legitimate business

No, you won't be. You're trolling the Internet and stealing other people's work and are wondering what problems might crop up. Let's see, there's possible litigation and in extreme cases perhaps even some time at one of your government's holiday camps. Laws vary from place to place but I'm prepared to bet that what you're doing, or planing to do, is not legal anywhere.

share|improve this answer
Couldn't have said it better myself. – Chopper3 Dec 2 '09 at 7:00
lol, google is not a legitimate business then. that's completely non sense.. it is %100 same as google, just more focused. Google has already indexed those type of documents. You either do not have any idea about internet or did not understand my question. I prefer the second one of course for the sake of this website. – Yuri Dec 2 '09 at 18:16
+1 I was about to say the same thing. Is Google not legit? How about Bing, Yahoo? Even better... what about :-) – KPWINC Dec 2 '09 at 19:11
Search engines provide links to content on the original web sites. They do NOT copy that content an place it on their own web sites. Perhaps some of you didn't see the actual question as it was originally posted. The OP was talking about taking other peoples work and hosting it on their own web site. THAT is the question I was responding to. – John Gardeniers Dec 2 '09 at 21:04
Search engines do reproduce and copy content. Try the "Cached" link on your results page on Google or Bing. – Stephen Jennings Dec 3 '09 at 0:22

Literally everything you would pull is under copyright from its authors. How they choose to license that content is up to them, but every work receives copyright.

You need to be careful.

share|improve this answer
+1 All copyrightable works are copyrighted by default, and you cannot redistribute that material unless you have some agreement or they license it in a way that allows it. – Stephen Jennings Dec 2 '09 at 5:44

You use the analogy that you will be operating in the same way as Google. I was under the impression that Google simply links to online resources. Your questions reads like you want to download and store copies of 3rd party resources within your own CDN.

Which is correct?

share|improve this answer
Oh sorry, google is not just linking, google is also caching the document.(So, you can look at the document through google docs if you want without going to the website) So we will also cache the document and show the cached document if the user wants. However, as you said we will also have a link to the source. We cannot now the source is the real owner or not. Just think, google is indexing the pdf files, some of the files are illegal e-books. So, if google get any complaints they remove. This is the same procedure as what we are planning to do. (however pdf files are not our focus) – Yuri Dec 2 '09 at 18:38
This is an interesting point. Google will serve pages from its cache if requested to do so. The implication being that they do download and store copies of some content. – Chris Thorpe Dec 2 '09 at 18:49
Also... what about OpenDNS Smartcache? Not only are they caching the page... they're "pretending" to be YOU!… – KPWINC Dec 2 '09 at 19:13
Thanks Yuri for clarifying that. Hmmm it does seem a legal grey area. However, think of the situation where an organisation operates an internal internet proxy, caching web requests. The requested website resources are being stored. Is this therefore illegal? You might want to research "take down request" procedures. – Ross Wilson Dec 3 '09 at 16:47
In addition to my last comment, I have just found the following resource: - it's a news article regarding Google's Cache and how a court have ruled it as legal. – Ross Wilson Dec 3 '09 at 16:48

Since even after your edit you refuse to specify what "focused" actually means, you'll have to contact the CDN provider in order to receive approval. My best guess is that you are in fact not doing what Google does, and you plan to cache more than simple or advanced text files.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.