You are seeing normal behavior. Don't confuse used memory with unavailable memory. Many structures held in memory can be quickly downsized when memory is required. Expect your usage to increase as long as the system is up and lots of memory is available.
Space used for disk cache counts as used. Once you read or write a file, the data tends to be cached until this space is needed. Log files tend to be write only, but get cached as they are generated. A few things that are noticeable from my usage graph:
- The longer a system is up the more memory is dedicated to file cache and buffers. This is limited by the amount of memory available. As memory requirements increase, memory dedicated to file cache and buffers may decrease as required.
- Total memory used stabilizes some Mb short of 100%. This is memory reserved for running sub processes. If this space gets too small expect slow performance and crashes.
- Total memory used is not nearly import as what it being used for.
If you have any tmpfs file systems, it will use available memory. As memory use goes up, some of this data may be paged into the swap file.
Web server processes often run with a master process to accept incoming connections, and many child processes to handle the requests. The child processes typically have the same PPID (parent process id). One of your servers appears to be running
php-cgi to handle web requests, using a significant number of processes. The command
pstree, if available, is useful to see this kind of setup.