Prometheus Data Management
Prometheus data is considered ephemeral, functioning more like a cache than a permanent store. For high availability, deploying a second Prometheus server can help mitigate reliance on a single machine, though slight differences in metrics between instances are generally acceptable. The inherent race conditions in monitoring mean that variations in data collection are expected and should be accounted for in a resilient monitoring strategy.In this clip
From this podcast

Software Engineering Radio - the podcast for professional software developers
SE-Radio Episode 270: Brian Brazil on Prometheus Monitoring
Related Questions