I was running three mongo (4.0.4) containers within docker with configured replica set over 10 months without having any problems or issues. Primary mongo container/replica set was connected with spring boot application that is running spring 2.1.1 with using mongo transactions.
At one point during the holidays all of my mongo db collections where gone, all the created collections and data within have disappeared. Since I have some scheduled process within spring boot and that process is inserting data into one collection that collection was created and data was inserted which is the correct way of functioning of this process (spring boot is automatically creating collections if they are not created already), so at the point where I saw I have lost the data I had only one collection and data that was inserted after the problem.
All three containers have shared volume from the host and when I was looking at the .wt files there is one file within some part of the data is readable and it seems that all dropped data is there but I don't know how could i restore them. In file collection-8--739794789310552254.wt there are around 3561507 lines and file is around 390mb. When I open this file in editor some part of the data is readable especially the dates (have audit log on all the documents, created, lastModified and etc).
Tried with mongo restore, no luck.
What could I only think how this happen since in the logs there where no logs about anybody accessing db and running any kind of drop statements I could only think of that some of the transaction within spring somehow locked and mongo returned to the last know state which was empty state (only reasonable explanation for me), why did it restored to empty don't know.
Is there some way to restore the data that is in collection-8--739794789310552254.wt I have the whole volume from /data/db path backed up and I'm able to recreate Mongo from that files with Docker but i have only the state after the problem.