I have Lotus domino Email Server 8.5.3 FP6. My question is can I run compact -a (and once the archive is done it should run compact -B) from command prompt on running domino (in off peak time, without stopping the domino services ?
It depends on which DBs you wish to run the command. What I normally do is to use a program Document to periodically run a server command as written here. And read also here if you wish to compact something. Both require that the server is online, otherwise the program document will not kick in. You couldn´t compact for example the admin4.nsf while the adminP task is running, but mail DBs should be working. I have done that multiple years.