The Story of a Mongo Collection

In my last company, we were having more than 70 small APIs and we were using MongoDB heavily. Apart from having ELK stack, we also had database level logging for all the incoming requests to some of the endpoints to track the requests and responses.

A few months ago, we were struck with a disaster, because of something foolish that was missed in the code reviews. One night we did a deployment and after doing the sanity checks on production, considering everything is fine, we all went home thinking that all is good.