Having numerous files will obviously slow down the OS
No, it really won't. I've had *NIX systems at 99% inode utilization ("near the upper limit on the number of files the filesystem can hold") and had no performance problems.
My workstation is currently at 90% inode utilization, and all my performance problems are due to insufficient RAM.
but how much this problem is serious? Consider that the files have been well distributed over multi-level folders. Does still the number of files (probably because of using inodes) can slow down the system?
This is not a serious problem. Properly architected you should be able to hit your system's inode limit without any performance probems.
Also note that every directory ("folder") uses an inode on *NIX systems.
I am talking about few millions! This is not too much for a Desktop computer, due to different programs; but this is too much for web servers.
On what do you base this (mostly incorrect) statement? Assuming they're running the same OS, why would your desktop and a server be magically different in terms of filesystem behavior?
"different programs" has no effect on filesystem performance. The operating system is responsible for telling you what files are where (logically within the filesystem and physically on the disk), and most filesystems are very efficient at this.
I am curious to know whether storing few millions of files (in appropriate folder) has a significant effect on the server performance?
Millions of files in one directory? Not advisable (and not possible on many systems -- there is usually a limit on the maximum number of files within a directory).
Walking a very large directory tree may cause performance issues (it takes the OS time to walk the tree and list all the children, and then your software has to deal with the giant pile of data it's being handed), but if you do not have a grossly unreasonable directory structure (like "Everything in
/dumping_ground") this should not be an issue.
In response to the edit above:
More Information: Consider ext4 as filesystem, and 100 files per folder in two-level folders.
You are joking, right? Consider the number of files in
# ls -a /usr/bin | wc -l
and that's small for