I have a flask app running in a docker container which scrapes data from few sites. For scraping I am using selenium and to run jobs in parallel I am using selenium/hub and selenium/chrome-node (varies on docker-compose scale value). The whole app is supposed to run on beanstalk worker tier as cron job.
Which beanstalk environment is best for the above situation?.
Plus, I have explored multi container docker. Can we scale up multiple chrome containers dynamically in multi docker ?
Plus, if both flask app and selenium/hub is running side by side as sibling containers. How flask app can access selenium/hub?