Repeat after me: Sharepoint is not a streaming media server
2010 may be a little better, but still...SharePoint is not a streaming media server.
File size limitations (yes, you can change it...no you shouldn't change it too much)
File handling. It won't 'stream' as you think it should. As you are playing a video, you will not be able to FF/REW as you can with, say, YouTube.
Backend DB throughput. SharePoint was not designed for this. It was really designed for small files. There is a testable drop in response as file size gets larger as opposed to a regular file system.
Yes, we've tried this. No, it doesn't work well, at least on SP 2007.
The recommended practice is to have the files stored in an regular old file system, with maybe a pointer from SharePoint.
But if you're looking to a) stay in the Microsoft world, and b) true streaming...go to Windows Media services on Server2008
Once more, for emphasis: SharePoint is not a streaming media server