Unit 3.4: Activity Logs and Monitoring

Introduction

Imagine running a large office building where you need to know three critical things: who entered which rooms and when (security access logs), what's happening in the ventilation system (network flow monitoring), and whether all the equipment is running smoothly (performance metrics). In the cloud, you need similar visibility into your infrastructure to maintain security, troubleshoot issues, and ensure everything operates as expected.

IONOS Cloud provides multiple observability services that work together to give you complete visibility into your environment. Activity Logs track every user action for security and compliance. Flow Logs capture network traffic patterns for connectivity troubleshooting. The Monitoring Service visualizes system performance metrics in real-time dashboards. The Logging Service centralizes application and infrastructure logs for analysis. Understanding what each service does and when to use it is essential for managing any cloud environment effectively.

1. Activity Logs for Audit Trails

Activity Logs provide a read-only, chronological record of every action performed on your IONOS Cloud resources. Think of them as a security camera system for your cloud account, recording who did what, when they did it, and from where.

1.1 What Activity Logs Capture

Activity Logs record a comprehensive range of events across your contract, creating an immutable audit trail that you cannot modify or delete. Every entry includes:

• User actions - Logins, logouts, and authentication events
• Resource operations - Provisioning new resources, configuration changes, deletions
• Data access - When users accessed or modified data
• Source information - IP addresses, services, or systems that initiated the action
• Timeline details - Precise timestamps for every event
• Affected resources - The specific resource type, ID, and the action taken

Because Activity Logs are read-only, you interact with them exclusively through GET requests via the API. This design ensures the logs cannot be tampered with, preserving their integrity for security investigations and compliance reporting.

1.2 Why Activity Logs Matter

Activity Logs deliver critical benefits for security, operations, and compliance:

Organizations use Activity Logs to answer critical questions like "Who deleted this snapshot?" or "When did a user log in from an unusual IP address?" The logs are filterable by date range and grouped by contract, making it easy to narrow down investigations to specific time periods or accounts.

1.3 Activity Logs in Practice

Contract owners and administrators access Activity Logs through the IONOS API. The logs are filterable by date ranges, allowing you to download entries for specific periods as JSON data. This makes it straightforward to integrate Activity Logs with security information and event management (SIEM) systems or to generate compliance reports for auditors.

Because logs are partitioned by contract, you see only the activity relevant to your specific resources, ensuring data isolation between different customers and accounts. This architecture supports multi-contract environments where administrators manage multiple customer accounts while keeping their audit trails separate.

2. Flow Logs for Network Visibility

Flow Logs capture detailed information about network traffic that passes through your virtual machines, Managed NAT Gateway, or Managed Load Balancers. Unlike Activity Logs that track user actions, Flow Logs show you the actual network communication patterns happening inside your infrastructure.

2.1 What Flow Logs Capture

Flow Logs record metadata about every packet flow that traverses a network interface. The following table shows what they capture and how it provides visibility:

The glossary defines flow logs as "a feature that allows you to capture data related to IPv4 and IPv6 network traffic flows, enabled for any network interface of a VM instance and Network Load Balancer, as well as the public interfaces of the NAT Gateway."

2.2 Why Flow Logs Matter

Flow Logs provide network-level observability that complements Activity Logs' user-level visibility. They help you:

• Monitor network security - See which connections are being blocked by firewall rules and identify potential security threats
• Troubleshoot connectivity - Diagnose why traffic isn't reaching its destination by examining flow patterns
• Audit firewall rules - Verify that security rules are working as intended by reviewing accepted vs. rejected traffic
• Detect anomalies - Identify unexpected traffic sources, unusual port usage, or sudden traffic spikes
• Support compliance - Many regulations require tracking and reporting network flows for data security audits

Once activated, a green indicator on the network interface properties shows the flow log rule is valid and provisioned, meaning data collection has begun. The flow data is stored as objects in IONOS Cloud Object Storage, where you control retention policies and can query the data using standard analytics tools.

2.3 Flow Logs in Practice

You create flow log rules for specific network resources (VM network interfaces, Managed NAT Gateway, Managed Network Load Balancer, or Managed Application Load Balancer). Each rule specifies:

• Name - Descriptive identifier for the rule
• Direction - What traffic to capture (Ingress, Egress, or Bidirectional)
• Action - Which packets to log (Accepted, Rejected, or Any)
• Target bucket - The IONOS Cloud Object Storage bucket where logs will be stored
• Optional prefix - Organizes logs within the bucket using a folder-like structure

The logs are written as objects to your specified bucket on a 10-minute rotation interval (no object is written for an interval with no traffic), and they remain there until you delete them or your bucket lifecycle policies remove them. This gives you complete control over retention periods while supporting long-term storage for compliance requirements.

3. Monitoring Service Overview

The Monitoring Service provides real-time visibility into system performance by collecting, aggregating, and visualizing metrics from your entire infrastructure. Unlike Activity Logs and Flow Logs that focus on security and networking, the Monitoring Service tracks the health and performance of your applications, servers, and services.

3.1 How the Monitoring Service Works

The Monitoring Service uses a centralized architecture that collects metrics from multiple sources and displays them in Grafana dashboards. The following table outlines what you can monitor:

All metrics are visualized in a single Grafana instance, giving you one place to query, explore, and analyze data from any source you push to the service. The Monitoring Service is designed to handle high volumes of metrics with near-real-time ingestion (approximately one-minute intervals by default), ensuring dashboards stay responsive even under heavy load.

3.2 Dashboard Customization and Alerts

The Monitoring Service offers powerful dashboard customization through Grafana. You can:

• Create new dashboards - Build custom views tailored to your specific monitoring needs
• Edit existing panels - Modify queries, change visualization types (graphs, tables, heat maps), adjust time ranges
• Share dashboards - Make dashboards accessible to other users or public within your organization
• Set up alerts - Define thresholds and conditions that trigger notifications via email, Slack, webhooks, or other channels
• Manage access - Use role-based access control to determine who can view, edit, or administer dashboards

Every sub-user gets at least a Viewer role, allowing them to see dashboards. Administrators can grant Editor or Admin roles to enable team members to create dashboards or configure alerts. Alerts are defined directly in Grafana panels by setting thresholds and conditions. When a metric breaches a rule, Grafana sends notifications through your configured channels.

3.3 Monitoring Service in Practice

To use the Monitoring Service, you create a monitoring pipeline in your desired region. The service returns a unique Grafana URL that remains consistent for all your monitoring pipelines. You then configure agents or exporters (Prometheus, Grafana Agent, OpenTelemetry, FluentBit) on your servers to push metrics to the regional endpoint.

Once metrics begin flowing, you open the Grafana URL and log in with your IONOS credentials. You'll see a default dashboard showing an overview of all metrics. From there, you can clone the default dashboard, customize it, and save your changes. Dashboards automatically refresh at configurable intervals, providing live visibility into your infrastructure's health.

The service supports flexible data retention policies, allowing you to store metrics for the periods required by your business or compliance needs. The default push interval is one minute, but you can adjust this based on your monitoring requirements and cost considerations.

4. Logging Service and Centralized Log Management

The Logging Service provides a centralized platform for collecting, storing, indexing, and visualizing logs from all your applications, containers, virtual machines, and infrastructure components. While Activity Logs track user actions and Flow Logs capture network traffic, the Logging Service handles application logs, system logs, and any other log data your infrastructure generates.

4.1 How the Logging Service Works

The Logging Service follows a shared responsibility model with two main components:

The architecture enables near-real-time log ingestion (typically within seconds), giving DevOps teams immediate visibility into what's happening across their infrastructure. Each log pipeline gets a unique regional endpoint, and every log source within a pipeline must use a unique tag so logs can be distinguished even when sharing the same endpoint.

4.2 Key Capabilities

The Logging Service delivers several essential capabilities for modern cloud operations:

The default ingestion rate is 50 HTTP requests per second per pipeline; this is a per-pipeline default that you cannot raise on demand, so higher throughput is achieved by creating additional pipelines (each with its own 50 requests-per-second quota) or by using the TCP transport, which supports roughly 10,000 log records per second. Each pipeline runs in its own partition, ensuring data isolation between users and preventing cross-contamination of log data.

4.3 Logging Service in Practice

Using the Logging Service involves several steps:

1. Create a pipeline through the Data Center Designer or API in your desired region
2. Install Fluent Bit on each host or container that produces logs you want to collect
3. Configure Fluent Bit with the pipeline endpoint host and port, a unique tag for the log source, and the SharedKey token
4. Start the agent to begin forwarding logs over TLS to the pipeline
5. Access Grafana to view, analyze, and create alerts on your log data
6. Set retention policies and configure sub-user access as needed

The Grafana interface lets you search logs, create dashboards combining log data with metrics from the Monitoring Service, and set up alerts based on specific log events. This integration provides a unified observability platform where logs and metrics appear side-by-side for comprehensive troubleshooting.

5. Choosing the Right Service

With four different observability services available, understanding when to use each is essential for effective cloud management. Each service focuses on a specific type of data and serves distinct operational needs.

5.1 Service Comparison

The following table summarizes what each service records, where data lives, and typical use cases:

5.2 Decision Guide

Use this guide to determine which service best fits your specific needs:

Many organizations use multiple services together to achieve complete observability. For example, you might use Activity Logs for compliance audits, Flow Logs to troubleshoot network connectivity issues, the Monitoring Service to track server performance, and the Logging Service to investigate application errors.

Common Use Cases

Real-world scenarios where these services work together:

1. Security Incident Investigation: A financial services company detects unusual database access patterns. They use Activity Logs to identify which user account was compromised, Flow Logs to trace the source IP addresses and connection patterns, and the Logging Service to examine application logs for signs of data exfiltration. The combined visibility from all three services enables them to contain the breach, revoke compromised credentials, and file a detailed incident report for regulators.

2. Application Performance Troubleshooting: An e-commerce platform experiences slow checkout page loads during a promotional event. The operations team uses the Monitoring Service dashboards (covered in Section 3) to identify elevated database query latency. They then examine the Logging Service (Section 4) for database error messages and discover a missing index. Flow Logs (Section 2) confirm that network connectivity to the database server is healthy, ruling out network issues. The team resolves the performance problem by adding the index.

3. Compliance Audit Preparation: A healthcare provider must demonstrate HIPAA compliance for their patient portal. They configure Activity Logs (Section 1) to track all user access to patient data, set retention policies in the Logging Service (Section 4.2) to keep application logs for seven years, and enable Flow Logs (Section 2.1) to document network access patterns. During the audit, they generate reports showing who accessed what data, when access occurred, and that all access was properly authenticated and authorized.

Summary

IONOS Cloud provides a comprehensive suite of observability services that work together to give you complete visibility into your infrastructure, applications, and user activity. Activity Logs create an immutable audit trail of user actions for security and compliance. Flow Logs capture network traffic patterns for troubleshooting connectivity and monitoring security. The Monitoring Service collects and visualizes performance metrics in real-time Grafana dashboards. The Logging Service centralizes application and system logs for analysis and long-term retention.

Understanding what each service does and when to use it is essential for effective cloud management. Activity Logs answer "who did what," Flow Logs reveal "what network traffic occurred," the Monitoring Service shows "how systems are performing," and the Logging Service helps you "troubleshoot application issues." Many organizations use multiple services together to achieve comprehensive observability across security, networking, performance, and application layers.

Key Points:

• Activity Logs provide a read-only, tamper-resistant audit trail of all user actions across your IONOS Cloud contracts, essential for security investigations and compliance reporting
• Flow Logs capture detailed network traffic metadata (source/destination IPs, ports, protocols, firewall actions) and store it in Object Storage for security monitoring and troubleshooting
• The Monitoring Service collects performance metrics from infrastructure, applications, and services, displaying them in customizable Grafana dashboards with alerting capabilities
• The Logging Service centralizes application and system logs using Fluent Bit agents, providing near-real-time visibility and long-term retention through a Grafana interface
• Each service serves distinct needs: Activity Logs for user actions, Flow Logs for network visibility, Monitoring Service for performance metrics, and Logging Service for application logs
• Organizations often use multiple services together to achieve complete observability across security, networking, performance, and application layers

Important Terminology:

• Activity Logs: Read-only audit trail recording every user action (logins, provisioning, configuration changes, data access) on IONOS Cloud resources with details about who, what, when, and where
• Flow Logs: Network traffic metadata capture that records source/destination IPs, ports, protocols, and firewall actions for VM network interfaces, Managed NAT Gateway, and Managed Load Balancers
• Monitoring Service: Centralized platform that collects performance metrics (CPU, memory, latency, error rates) from infrastructure and applications, visualized through Grafana dashboards
• Logging Service: SaaS-based platform that aggregates application and system logs from multiple sources into a single searchable repository accessible via Grafana
• Grafana: Open-source visualization and analytics platform used by both the Monitoring Service and Logging Service to display dashboards, create alerts, and explore data
• Fluent Bit: Lightweight log forwarding agent used by the Logging Service to collect logs from various sources and send them securely to the aggregation platform

Next Steps

Continue Learning: Unit 3.5: Security and Compliance

Related Topics:

• Unit 2.4: Networking Services
• Unit 3.2: Identity and Access Management
• Unit 3.3: Cost Management and Billing