This is my use case:
I have a decoupled application system. One application is accepting requests, validating, and queuing in SQS. Other application is mounted on a cluster of EC2 workers that will process messages. After a first processing I need to queue again the result in a second layer (persistent queue) for auditing/further processing and archiving activities (with retention period)
For this second persistent queue I chosed for my convenience RDS. Now I'm evaluating (for an architecture improvement) to switch to DynamoDB.
Since: - I don't actually need relational features in my applications - I wouldn't want to manage resizing instances / high availability - I know DynamoDB offers recordset TTL - It will be more cost-effective? (since this persistent queue will be infrequent access)
But: - From my management UI I'll need to give the chances to scan the queue (pagination, search etc...) and I know these are weak points for Dynamo - Management application /and workers are developed with Laravel and Eloquent Orm (that integrates perfectly with MySQL e.g. but not with Dynamo if not with third parties composer dependencies) will presumably need app re-writing
What do you recommend? Is there some other chance I don't see? Thanks!