We're building an API-based web solution on Python which is going to be delivered to individual customers. Each customer is going to have a separate dataset.
Currently, it's not a problem to deploy an instance of the application + database for each customer, but this is going to become nightmare with more customers: imagine hundreds of such instances and administration expenses on that. Obviously, the solution should become more "centralized".
However, the data is quite sensitive, so we can't keep everything in a single database: a compromized server and DB password will lead to the leakage of all data. Such risks are unacceptable.
What are the right technologies to consider when developing a distributed API server platform? Which smart practices will make the solution more "centralized" so we get rid of that many instances but still keep the data isolated?