How to subscribe a specific instance within an elastic beanstalk application to an SNS topic?

Question

Ok, so I have an elastic beanstalk application with a scalable web tier, served behind an ELB. Long story short, I need to be able to subscribe a specific instance within my web tier to an SNS topic. Is it safe for me to use a standard method to get the instance ip address (as detailed in python here How can I get the IP address of eth0 in Python?) and then simply subscribe to an SNS topic using that ip as an http subscriber?

Why? Good question...

My datamodel is made up of lots of objects many of which can have an attached set of users which may want to be able to observe those objects. This web tier in my application is responsible for handling the socket interface (using socket.io) for client applications.

When a user is created in the system, so too is an SNS topic for the user, allowing notifications to be pushed to that user when an object it is interested in changes. The way I am planning to set this up, a client application will connect to EB via socket.io at which point the server instance it connected to will subscribe to that user's SNS topic. Then when an interesting object changes, notifications will be posted to the associated user's topics, thus notifying the server instance that the client application has an open connection to, which can then send a message down the socket.

I believe it is important that the specific instance is subscribed rather than the web tier's external CNAME or ip, as the client application is connected to a specific instance and so only that instance can send messages over it's socket. Subscribing the load balancer would be no good as the notification may be delivered to an instance that the user is not connected to.

I believe the question at the top is all I need, but I'm open to creative solutions if my reasoning seems flawed??

Answer 1

This architecture is not well suited for your use-case.

1. Creating an SNS topic "per user" won't scale,
2. Having your backend EC2 instances subscribed requires those backend EC2 instances to have public endpoints,
3. As EC2 instances get created and terminated, stale subscriptions will stick around, and have to be managed/deleted,
4. Also, locking a user to a specific EC2 instance won't work well if that EC2 instance were to be terminated (due to scaling down or other failure).

Instead, try:

1. Allow your client connections to connect to the load balancer, which then allows connections to jump EC2 instances as needed.
2. Use another mechanism, such as Redis pub/sub, to manage getting messages to the EC2 instances, and thus to the clients.