3

On one server I have some 30 PHP sites running under Apache. All those sites use the same (HTTP) API to fetch some data. The API is hosted elsewhere (under my control)

The API uses Nginx with keep-alive and the PHP sites use CURL for making the API requests.

A visitor to 1 of the 30 sites would generate an API call, and the connection to the API would be closed by apache/PHP as soon as the HTML is delivered to the visitor.

What I'm looking for is something like a local proxy to the API that is able to maintain the connection to it so the PHP sites can profit from the keepalive.

Anyway of accomplishing this?

3 Answers 3

1

Nginx configured as a reverse proxy can do this easily :

http {
 upstream remoteserver {
  # here you add your remote server's IPs or hostnames
  server 54.175.222.246; # for example here we use HTTPBin's address
  keepalive 10; # maintain a maximum of 10 open connections
 }

 server {
  listen 80;

  location / {
   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # passing the client's IP to the remote server, on a local machine it doesn't do much though
   proxy_set_header Host $http_host; # passing the Host header as requested by the client, otherwise this will default to the pool's name, "remoteserver" in this case
   proxy_pass http://remoteserver; # sends the request off to the pool defined above
  }
 }
}

Now you can point your scripts to the local server instead of the remote one, here's a demo with curl :

$ curl http://localhost/get -H "Host: host header is passed normally"
{
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "host header is passed normally", 
    "User-Agent": "curl/7.29.0"
  }, 
  "origin": "127.0.0.1, 1.2.3.4", 
  "url": "http://host header is passed normally/get"
}

As you can see, even the Host header is passed as-is.

Or you can make the transition seamless by making the remote hostname point to your local machine, either in /etc/hosts or in your DNS resolver's configuration. In this case, make sure to use only IP addresses instead of hostnames in the pool definition in Nginx's config, otherwise the proxy will also loop back to itself and that would cause a bit of a disaster.

Once the hosts file has been changed accordingly, the proxying is seamless :

$ curl http://httpbin.org/get -v
* About to connect() to httpbin.org port 80 (#0)
*   Trying 127.0.0.1...
* Connected to httpbin.org (127.0.0.1) port 80 (#0)
> GET /get HTTP/1.1
> User-Agent: curl/7.29.0
> Host: httpbin.org
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx/1.6.2
< Date: Sun, 15 Mar 2015 00:41:54 GMT
< Content-Type: application/json
< Content-Length: 198
< Connection: keep-alive
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
< 
{
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.29.0"
  }, 
  "origin": "127.0.0.1, 1.2.3.4", 
  "url": "http://httpbin.org/get"
}

As you can see, our local server behaves just like the remote one and any program attempting to access the remote hostname will actually connect to our local server.

Note that this may need additional configuration for HTTPS-based hosts.

0

PHP's socket functions may be the easiest approach. Socket_create will handle IPv4, IPv6, and UNIX connections. A quick example

$socket = socket_create(AF_INET,SOCK_STREAM,SOL_TCP);
$address = '127.0.0.1';
$port = 80;
socket_connect($socket,$address,$port);
// Sending data
socket_write('Your API commands here');
// Reading data
while ($buffer = socket_read($socket,1024,PHP_NORMAL_READ)) {
  if(trim($buffer) == 'END') {
    break;
  }
}
socket_close($socket);

More examples at php.net: Socket Examples

1
  • 1
    Thanks for your answer! But since the sites can generate multiple API requests per second, i don't think a single threaded PHP process running as proxyserver would be a reliable solution..
    – Ivor
    Oct 12, 2012 at 6:37
0

You might want to write a php script that would be launched from command-line and would demonize, open a curl handle and would reuse it for every subsequent request therefore using the keep-alive feature. This script should provide an API using messaging queues (check out beanstalkd / rabbitmq). As soon there's a new message in the queue, the script should make a request to the external API and push the result back into the messaging queue. Alternatively provide an API using sockets (but this might be very tricky since it should be multi-threaded, multi-multithreading in PHP might be achieved using fork and I'm not sure how the curl handle would behave if you'd try using it in multiple subprocesses at a time). There might also be a performance hit for using this so if you have many users you probably should create multiple daemons that run at the same time.

2
  • 1
    Pretty overkill for something that can be done way more reliably with an Nginx configured as a reverse proxy and the local scripts pointing at the local Nginx instead of the remote server (spoof its DNS if you can't force your users to update their scripts).
    – user186340
    Mar 15, 2015 at 0:19
  • Totally agreed. I just didn't know nginx is capable of reusing the same connection. +1 for nginx :)
    – s3v3n
    Mar 15, 2015 at 1:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .