Local Grafana - Log Strategy

[raftmsohani]

Description:
We need to specify how we would like to logs backend logs. We also need to add frontend, server level health monitoring, and Postgres logs to Prometheus.

All modifications in this issue are limited solely to the LOCAL ENVIRONMENT but should be environment agnostic, ensuring that we can fulfill all requirements and acceptance criteria in this issue on cloud.gov in next ticket.

Acceptance Criteria:

As an engineer I want to create and improve log strategy for sending logs to Prometheus

• Logs from backend are available in Grafana
• Logs from frontend (Nginx) are available in Grafana
• Backend and Frontend servers are monitored by Grafana (availability, cpu usage, HDD usage)
• README is updated, if necessary

Tasks:

• Add/configure Promtail to compose and define scrape endpoints (frontend and backend)
• Add/configure Loki
• Build Grafana dashboard for logs
• Run Testing Checklist and confirm all tests pass

Supporting Documentation:
Please include any relevant log snippets/files/screen shots

• An example for logging Django logs using Prometheus
• Monitoring Django app with Prometheus

Open Questions:

• What other metrics do we want to add to Grafana?