A) run a check on your system with ClamAV to see if anything out of the ordinary is detected.
B) Run rootkit checkers to see if they find anything out of the ordinary.
C) Run a packet sniffer (tcpdump, wireshark) on your server in question to see if there's unusual traffic running through it.
D) check /tmp, /var/log, etc. to see if there are any unusual temp files or logfiles deposited there.
E) happen to have any programs like tripwire installed to look for unusual changes to system files?
F) Check with other admins to see if they've been doing something with that server that could trigger a check on the remote system.
What exactly does the log SAY is happening? Requesting a particular file? Just pinging it? Are they running a listening process on 443 that logs what is happening, or do they have a legit process handing requests on that port? You can narrow down a little bit (or eliminate other possibilities) by looking at what is in the request. Also, is your server's IP showing up somewhere else in their logs, like requesting non-SSL web pages or something like that?
Is your system scanning other machines in your network? Is the IP showing up for unusual activity in other server's logs?
You can also set up something like Snort or Honeyd within your network for continued monitoring to see if something's going on. My advice on the clam and rootkit checkers are only half-solutions since if your system IS compromised, only an offline check may actually find malware if the system binaries have been compromised (unless you have another server of the same configuration against which you can run md5 checksums on certain binaries to see if they've been tampered with).