I've got an Apache server hosting several json APIs.
This server is running Ubuntu 16.04.4 LTS with Apache and Mysql.
Response times for all API are very good except for one! It takes about 5s to get a json response from the particular API whereas it takes milliseconds for the other APIs.
I checked multiple things without success :
1) the SQL queries ran almost instantly (with and without cache - I tried both)
2) our developper added a custom header to find out how long the API takes to generate the json response : 2-3 ms
3) calling the API from a remote computer or locally on the server with wget gives the same result : it takes 5s to get the reply. According to the browser developper mode, the 5s are spent "waiting". Google Chrome says "Waiting for the first byte". Then data transfer seems fast according to the dev mode (a few ms)
4) apache doesn’t seem to be overwhelmed by https requests, see screenshot displaying server-status page Server status
5) access to the https API is done through multiple firewalls but no reverse proxy is present
As part of the debugging process, I tried to improve the server performances :
I moved the vm to a our least busy VMware host
I moved this vm to an SSD based datastore
Do you have any idea why Apache is so slow to reply ? Do you know any way to troubleshoot this issue ?
Thank you for your help