Integrating Datadog with Helm Charts for SSL-Secured External Database Monitoring
I’ve been tasked with setting up monitoring for an external database using Datadog. This database, hosted as a service and secured with an SSL certificate provided by Azure, requires careful integration. Below are the steps to accomplish this effectively.
Great, here’s the requirements for our integration :
1- database to monitor -> I’ll use MySQL as a Database-as-Service.
2- A Kubernetes cluster -> I’ll use Azure Kubernetes Service (AKS).
3- Helm installed for chart management
Step 1: Setting Up Database Monitoring for Azure Database for MySQL
Database Monitoring provides deep visibility into your MySQL databases by exposing query metrics, query samples, explain plans, connection data, system metrics, and telemetry for the InnoDB storage engine.
The Agent collects telemetry directly from the database by logging in as a read-only user. Complete the following steps to enable Database Monitoring with your MySQL database:
Grant the Agent access
The Datadog Agent requires read-only access to the database in order to collect statistics and queries.
Create the datadog
user and grant basic permissions:
CREATE USER datadog@'%' IDENTIFIED by '<UNIQUEPASSWORD>';
ALTER USER datadog@'%' WITH MAX_USER_CONNECTIONS 5;
GRANT REPLICATION CLIENT ON *.* TO datadog@'%';
GRANT PROCESS ON *.* TO datadog@'%';
GRANT SELECT ON performance_schema.* TO datadog@'%';
CREATE SCHEMA IF NOT EXISTS datadog;
GRANT EXECUTE ON datadog.* to datadog@'%';
GRANT CREATE TEMPORARY TABLES ON datadog.* TO datadog@'%';
Create the explain_statement
procedure to enable the Agent to collect explain plans:
DELIMITER $$
CREATE PROCEDURE datadog.explain_statement(IN query TEXT)
SQL SECURITY DEFINER
BEGIN
SET @explain := CONCAT('EXPLAIN FORMAT=json ', query);
PREPARE stmt FROM @explain;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END $$
DELIMITER ;
Additionally, create this procedure in every schema from which you want to collect explain plans. Replace <
mydatabase>
with your database schema:
DELIMITER $$
CREATE PROCEDURE <mydatabase>.explain_statement(IN query TEXT)
SQL SECURITY DEFINER
BEGIN
SET @explain := CONCAT('EXPLAIN FORMAT=json ', query);
PREPARE stmt FROM @explain;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END $$
DELIMITER ;
GRANT EXECUTE ON PROCEDURE <mydatabase>.explain_statement TO datadog@'%';
Step 2: Setting Up Your Kubernetes Cluster
Ensure that your Kubernetes cluster is up and running. If you’re new to Kubernetes, you can find many resources online for setting up a cluster on various cloud platforms or on-premise environments.
Step 3: Installing Helm
If you haven’t already, install Helm on your system. Helm will be used to deploy the Datadog agent on your Kubernetes cluster. For detailed instructions on installing Helm, visit the official Helm documentation.
Step 4: Adding the Datadog Helm Repository
Add the Datadog Helm repository to your Helm client using the following command:
helm3 repo add datadog https://helm.datadoghq.com
helm3 repo update
Step 5: Configuring the Datadog Agent
i’ve cloned the chart helm from github
https://github.com/DataDog/helm-charts/tree/main
git clone https://github.com/DataDog/helm-charts.git
cd /home/href/datadog/helm-charts/charts/
update a values.yaml
file to specify the configuration of the Datadog agent. The crucial part here is to configure the agent + clusterAgent to monitor your external database. This can involve :
5–0 : require configs : api-key , site datadog,
5 —1 : add option DD_SKIP_SSL_VALIDATION on Cluster-Agent & Agent
## The Cluster-Agent supports many additional environment variables
env:
- name: DD_SKIP_SSL_VALIDATION
value: "true"
5 — 2 create a secret in order to use DigiCertGlobalRootCA.crt.pem
Just download the certificat from azure :
https://learn.microsoft.com/en-us/azure/mysql/flexible-server/how-to-connect-tls-ssl
- create a secret kuberntes usgin the command bellow
kubectl create secret generic my-ssl-cert --from-file=DigiCertGlobalRootCA.crt.pem=DigiCertGlobalRootCA.crt.pem
5–3 : add a volumes and volumeMounts on both clusterAgent.volumes & agents.volumes in order to copy DigiCertGlobalRootCA.crt.pem inside /etc/ssl/certs
# clusterAgent.volumes -- Specify additional volumes to mount in the cluster-agent container
volumes:
- name: ssl-cert
secret:
secretName: my-ssl-cert
volumeMounts:
- name: ssl-cert
mountPath: /etc/ssl/certs
readOnly: true
# agents.volumes -- Specify additional volumes to mount in the dd-agent container
volumes:
- name: ssl-cert
secret:
secretName: my-ssl-cert
# agents.volumeMounts -- Specify additional volumes to mount in all containers of the agent pod
volumeMounts:
# - name: <VOLUME_NAME>
# mountPath: <CONTAINER_PATH>
# readOnly: true
- name: ssl-cert
mountPath: /etc/ssl/certs
readOnly: true
5–4 : Set mysql configuration on Cluster-Agent
Step 6: Deploying the Datadog agents
helm3 install datadog-release datadog/datadog -f values.yaml
Step 7: Verifying the Setup
Once the Datadog agent is deployed, verify that it is properly collecting data from your external database. You can do this by checking the Datadog dashboard for incoming metrics and logs from your database.
. This setup ensures you have a robust monitoring system in place, providing you with real-time data and alerts to keep your systems running smoothly.
i hope this will help ,If you have any further questions or run into issues, don’t hesitate to reach out.
if you like my article :
NHAILA Achraf
#devsecops #devops #dockerswarm #swarm #cluster #datadog #azure #monitoring