After digging in Apple's Support forums I found a link to an article by David LeBer about this issue:
According to posts in Apple’s OS X Server forum (here, here, and the tail end of here) it looks like this nasty problem involves either servermgrd and/or DirectoryService. The number of open reference continuously grows until it renders your server unusable – usually not considered a good thing.
He provides some pointers to discussion forums and a "bandaid" via a script that looks for the infamous "Potential VM" notice on the server logs and restarts the affected service.
For future questions, it would be easier to help you if you provide more details about the platform you are running: OS version, roles of the server, number of users, concurrency...