7

We're developing an new server to replace an old one, and want to put the side by side for before & after tests - so that we can compare the end result and find the discrepancies.

The input to both services (old and new) is HTTP requests (mostly if not all GET, but possibly also POST), we need to have traffic coming to the old server duplicated and sent also to the new server (not instead). of course we will need to do some slight rewriting on the Hostname and port in the request to avoid an endless loop.

Old server is running on Apache (PHP), new one is Jetty (Java, Dropwizard)

There must be a way to do this, I just can't find the keywords for the Google search I guess...

  • 4
    Which response do you think you will want to return to the client? Because if you're copying all the data to the servers, then both servers will return data... Having proper testing requires a lot more than just copying data streams like this. – Jenny D Jan 23 '14 at 9:47
  • 2
    @JennyD I do not want any response to the user from the new server... that is a good point, and one we thought about (this is what made it impossible to do at the load balancer level with TCP/IP). – Moshe Eshel Jan 23 '14 at 13:46
  • @Iain I know about the option to replay captured traffic, that can also be done with replaying the access logs using JMeter, but this is something we will use only if the original option of duplication is impossible... – Moshe Eshel Jan 23 '14 at 13:48
  • 3
    @MosheEshel Replaying captured traffic is pretty much the standard for this kind of testing and it's definitely what I would advise. – Jenny D Jan 23 '14 at 13:49
2

What we chose eventually was using Gor (now GorReplay) https://github.com/buger/goreplay

this solution allows installing a listener on the original host, and what it does is record any incoming HTTP request, this is done without modifying it or blocking the production server from handling it.

It then pushes these requests to a Gor replay server which can handle all kinds of useful logics of splitting/increasing load based on the incoming requests - you can send a percentage of requests to a dev server, or a multiplication of the requests to create simulated (but from real traffic) load on your staging environment, or both...

Sadly this is at the server level, so you have to install on each production server to get all the traffic, but you don't have to, and it provides a great solution for the problem outlaid in my question.

1

Although is not what you are asking for, I will suggest another approach to test the new server.

If you put a load balancer in front of both servers and play with the load balancing algorithms you can at the same time test the new server and gradually replace the old one. You can send 99% of the requests to the old server and the remaining one percent of requests will go to the new one where you can closely review if the service is working as expected.

If everything works fine you can increase the load gradually; 90%-10%, 80%-20%, and so on.

Hint: Check haproxy and the weight and static-rr options.

  • 1
    We considered this, but we can't yet deploy the back-end for this, so the traffic sent to the new server will be effectively lost - and we can't afford that, even on a level of 0.001% – Moshe Eshel Jan 23 '14 at 13:49
1

There is another tool written in NodeJS and it's free https://www.npmjs.com/package/duplicator or in github https://github.com/agnoster/duplicator

As now passed too much time since the question at least hope this could help other readers if not to the original requester.

0

This is probably late but for those who will need it later;

You can use mitmproxy to do that. It allows duplication and modification (changing the host in your case) of the captured requests. In fact, there is an example about this in github repo. I used it in client side, but from my understanding, it can be used at the server side by setting up a reverse proxy for your old server.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.