ServicesTechnologies

Datadog: A Brief Overview of Monitoring Platform

By PlaysDev
Published: Nov 21, 2023

The trend towards increasing complexity of systems and applications is driving the adoption of more effective monitoring tools. In this context, Datadog stands out as a powerful and versatile tool.

What are the main features of Datadog and why do companies choose this paid SaaS monitoring in the world of open source and free software? Let’s figure it out.

The main factors we focused on are:

  • Price
  • Scalability
  • UI and Ease of Use
  • Functionality
  • Customization Capability
  • Predictable Usage
  • Deployment Simplicity

We will try to elaborate on each of them. Let’s start with the features of Datadog that have kept it at the top of the monitoring tools for several years and also conduct a brief comparison with the Prometheus-Loki-Grafana stack.

Easy Initial Setup (Low Entry Threshold)

For monitoring, it is enough to install an agent on the host and/or add the library in the application. The user-friendly interface and dashboards with key metrics save a significant amount of time at the initial stage. Moreover, Datadog has simplified integration with Microsoft Azure, making it even more attractive compared to Prometheus-Loki-Grafana. The latter requires initial configuration for collaboration and additional setup of exporters.

Dashboard Creation for Applications, Environments, and Custom Metrics in One Panel

Drag-and-drop widgets allow the creation of custom views without the need for coding. The visualization toolset allows data to be viewed in various formats and enables report creation.

A significant advantage is the ease of query creation, unlike Prometheus. In Datadog, everything is quickly and easily configured through the UI (often, Datadog even suggests potential queries, or metrics that already provide the required information, such as cpu_usage – a metric that returns the percentage of CPU usage).

Database Monitoring

Datadog Database Monitoring supports both self-hosted and cloud versions of Postgres, MySQL, Oracle, and SQL Server. The “Query Metrics” dashboard displays the historical performance of normalized queries. It allows visualizing performance trends using infrastructure or custom tags, such as data center availability zones, and provides alerts for anomalies.

Datadog also supports functionalities like:

  • Identifying queries that take the most time.
  • Tracking database-level metrics not captured by APM, such as updated/returned rows.
  • Filtering and grouping queries based on arbitrary parameters like command, user, cluster, and host.
  • Detecting unusually slow but rare queries not captured by metrics.
  • Assigning the execution of a specific query to a user, application, or client node.

A key feature is Datadog’s ability to monitor cloud versions of databases, a capability absent in PLG. In such cases, separate services like AWS CloudWatch or Azure Monitor would be necessary.

Serverless Stack Monitoring

Datadog provides solutions for monitoring AWS Lambda, Azure App Service, Azure Container Apps, and Google Cloud Run with features including:

  • Real-time alerts on memory, timeout, and parallelism metrics to prevent degradation of end-user service quality.
  • Tracking microservices calls for end-to-end visibility into client requests.
  • Visualization of distributed microservices on a service map, categorized by tags such as function, client, version, etc.
  • Receiving and analyzing 100% of traces over the last 15 minutes.
  • Isolating individual client requests and transitioning to related logs and metrics for a complete history.
  • Monitoring anomalies, outliers, and forecasting based on machine learning.

Kubernetes (K8s) Monitoring

This point is noteworthy as Datadog integrates with Kubernetes, Docker, containerd, and Istio, allowing:

  • Collection of metrics, events, and logs from cluster components, pods, and other Kubernetes objects.
  • Gathering container-level metrics for detailed resource breakdowns (at the Docker and containerd levels).
  • Datadog’s agent automatically tracks Kubernetes cluster nodes.
  • The Datadog automatic detection system, along with over 650 built-in integrations, automatically monitors technologies you deploy.
  • APM and distributed tracing provide an understanding of applications running in Kubernetes clusters at the transaction level. Datadog ensures monitoring at various levels of infrastructure, thanks to the installed agent—a feature PLG lacks, as it requires deploying Helm charts, installing kubernetes-event-exporter, and configuring all of it.

Unified Monitoring

Datadog offers robust monitoring capabilities for real-time tracking of various components’ performance. This includes server status monitoring, network activity, application response times, etc.

  • Log collection: Datadog allows organizations to centralize and analyze log data, aiding in troubleshooting and pattern identification.
  • Real-time monitoring: The real-time monitoring feature provides instant updates on performance and system status.
  • API monitoring: Datadog enables tracking of APIs to ensure their availability and responsiveness.
  • Response time tracking: The platform provides information on application response times, optimizing user experience.

Synthetic Monitoring

Synthetic tests allow observation of how systems and applications operate using modeled queries and actions from around the world. Datadog monitors the performance of web pages and APIs from the server-side to various network levels (HTTP, SSL, DNS, WebSocket, TCP, UDP, ICMP, and gRPC) in a controlled and stable manner, alerting about malfunctions.

  • Calculating SLOs on key endpoints and user routes simplifies adherence to application performance targets and ultimately ensures stable customer service quality.
  • Synthetic tests can be created within the Datadog application, using the API, or with Terraform.

Despite all the capabilities of Datadog, there are aspects that should not be overlooked when choosing and implementing this solution.

Complex Log Reception, Indexing, and Storage Process:

The log analysis process in Datadog is more complex than it should be. While you can send logs to Datadog, analyzing them proves challenging. If you want to analyze logs, you need to index and store them, which comes with a separate pricing structure for reception and storage. Due to complexity and cost structures, some organizations prefer not to store as many logs as they might need or want. This leads to challenges in troubleshooting and root cause analysis, especially for persistent issues that continue beyond the storage period.

To index and analyze logs, they need to be extracted from cloud object storage (e.g., Amazon S3) and reanalyzed. This process can take several hours and requires someone to manage it. Due to a constant shortage of specialists and an excess of work for DevOps and site reliability teams, many organizations cannot handle this level of complexity.

Expensive Log Analysis Workflow:

Regarding logs, Datadog charges $0.10 USD for data ingestion and from $1.06 USD (3 days) to $2.50 USD (30 days) for storage. To store logs for a longer duration, you need to contact Datadog and negotiate individual prices, which can quickly escalate as your company scales. While Datadog is useful for monitoring and detection, when it comes to root cause analysis and issue resolution, these costs can quickly spiral out of control.

Scaling Issues:

Reducing log retention periods can be a significant compromise, resulting in a loss of visibility into more complex issues—from prolonged performance problems with applications and infrastructure to persistent security threats. Many startups, starting with Datadog, discover that as they scale, they eventually spend absurd amounts of money on log retention. With scaling, Datadog becomes more expensive and challenging to use. Datadog has made a name for itself as a monitoring tool for startups due to its quick and easy setup. However, as projects grow, it becomes more expensive and challenging to use.

However, the ability to monitor serverless applications, k8s clusters and databases, and the use of AI in analyzing metrics and logs, makes Datadog undoubtedly one of the most modern and popular SaaS applications.

You may also like

Expertise
2024-03-15
PlaysDev
Top 8 Project Manager Skills: hard and soft skills to put in your resume
A short guide to the Project Manager profession: who is he and what responsibilities does he perform, what skills should a valuable PM have and how to develop them?
Читать
Expertise
2024-07-31
PlaysDev
OKR vs. KPI – Which metrics should you choose for IT projects?
Guide to choosing metrics for IT projects: we talk about different approaches to managing achievements and results. Will be useful for Project Manager.
Читать
Industries
2024-04-24
PlaysDev
DevOps Engineer in Fintech – Role, Tasks, 2024 Market Features
Main features of IT projects in the fintech industry review: What areas of development are popular in 2024 and what awaits us next? How can a DevOps engineer be useful on a fintech project?
Читать
Expertise
2024-09-18
PlaysDev
How corporate culture helps employees and managers succeed
Everyone has heard of corporate culture, but what does it mean in practice? What is behind the vague concept of “culture”? We talked about the main tools for achieving interest and involvement of employees.
Читать
Industries
2024-03-12
Dmitry Ostroga
IT Conferences for business: What’s beneficial about it and where to find one
Learn where to find the most impactful IT conferences, whether through global platforms or specialized niche gatherings. Elevate your business's IT strategy and stay ahead of the curve with the insights shared in this comprehensive article.
Читать
Expertise
2023-12-21
PlaysDev
Who is a Business analyst?
Who is a business analyst and what does he do in the company? What benefits does it bring to the company? Read about it in our article.
Читать
Expertise
2023-11-17
PlaysDev
Tips for a Successful IT Interview
In this article, we will look at how to prepare and successfully pass an interview in an IT company
Читать
ServicesTechnologies
2023-11-21
PlaysDev
Datadog: A Brief Overview of Monitoring Platform
In this article I will look at the Datadog platform: its advantages and disadvantages, entry threshold, types and monitoring systems, and much more...
Читать
Expertise
2024-09-24
PlaysDev
NDA: How to Protect Your Business from Confidentiality Breaches
How to avoid information leaks and protect your business? Read the article to learn how to draw up an NDA, what points should be included, and what mistakes companies most often make.
Читать
Technologies
2024-04-10
PlaysDev
Complete Guide To IoT technology: Internet Of Things in simple words
We describe the Internet of Things market using examples of popular IoT devices. What is IoT technology and what does it have to do with things: we tell you where the concept of the Internet of Things came from and how it is used now.
Читать