OpenTelemetry custom logs

Send your custom logs to Coralogix using our OpenTelemetry-compatible endpoint.

Overview

Custom logs can be delivered directly to the Coralogix OpenTelemetry-compatible ingress.:443 endpoint using any gRPC client, plain HTTP requests (OTLP/HTTP with JSON or binary protobuf payloads), or OpenTelemetry SDKs.

The examples below guide you using gRPCurl, cURL (OTLP/HTTP with JSON), and the OpenTelemetry Java SDK.

Prerequisites

Git and gRPCurl installed.

Data model

The custom logs API implementation is based on the OpenTelemetry logging specification. This ensures that our logging implementation adheres to industry best practices and can seamlessly integrate with other components and tools in the OpenTelemetry ecosystem.

Example log

{
  "resource_logs": [
    {
      "resource": {
        "attributes": [
          {
            "key": "cx.application.name",
            "value": {
              "string_value": "my-test-application"
            }
          },
          {
            "key": "cx.subsystem.name",
            "value": {
              "string_value": "my-test-subsystem"
            }
          }
        ]
      },
      "scope_logs": [
        {
          "scope": {
            "name": "test"
          },
          "log_records": [
            {
              "time_unix_nano": "1665989944490035000",
              "severity_number": "SEVERITY_NUMBER_WARN",
              "severity_text": "WARN",
              "body": {
                "string_value": "Test log message"
              }
            }
          ]
        }
      ]
    }
  ]
}

Sending data with gRPCurl

gRPC is a modern way of calling APIs on top of HTTP/2. Similar to cURL, gRPCurl is a command-line tool used to communicate with gRPC services.

• For the <open-telemetry-endpoint> field, select the ingress.:443 endpoint that corresponds to your Coralogix domain using the domain selector at the top of the page.

• For the <send-your-data-api-key> field, input your Coralogix Send-Your-Data API key.

Assuming the example in the data model is saved as logs.json, use the following command to send your data to Coralogix:

# Clone OpenTelemetry protobuf definitions
git clone https://github.com/open-telemetry/opentelemetry-proto.git
# Send logs to Coralogix 
grpcurl -v -d @ \
  -rpc-header 'Authorization: Bearer <send-your-data-api-key>' \
  -proto opentelemetry-proto/opentelemetry/proto/collector/logs/v1/logs_service.proto \
  -import-path opentelemetry-proto \
  ingress.:443 \
  opentelemetry.proto.collector.logs.v1.LogsService/Export \
  < logs.json

Sending data with cURL (OTLP/HTTP)

The same endpoint also accepts OTLP/HTTP requests on the /v1/logs path, with the payload encoded either as JSON (Content-Type: application/json) or as binary protobuf (Content-Type: application/x-protobuf).

The JSON payload follows the OTLP/JSON mapping, which differs from the gRPCurl example above:

• Keys are lowerCamelCase (timeUnixNano); original snake_case proto field names are not valid and are ignored.
• Enum fields are integers (for example, severityNumber: 13 instead of "SEVERITY_NUMBER_WARN").
• 64-bit integers are decimal strings.

Save the following example as logs-http.json:

{
  "resourceLogs": [
    {
      "resource": {
        "attributes": [
          { "key": "cx.application.name", "value": { "stringValue": "my-test-application" } },
          { "key": "cx.subsystem.name", "value": { "stringValue": "my-test-subsystem" } }
        ]
      },
      "scopeLogs": [
        {
          "scope": { "name": "test" },
          "logRecords": [
            {
              "timeUnixNano": "1665989944490035000",
              "severityNumber": 13,
              "severityText": "WARN",
              "body": { "stringValue": "Test log message" }
            }
          ]
        }
      ]
    }
  ]
}

Then send it to Coralogix:

curl -v "https://ingress.:443/v1/logs" \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer <send-your-data-api-key>' \
  --data-binary @logs-http.json

Sending data using the OpenTelemetry Java SDK

The example below guides you using OpenTelemetry Java SDK to send your custom logs to Coralogix. Others SDKs may also be used.

STEP 1. Add to your maven pom.xml the following libraries:

<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-sdk-logs</artifactId>
  <version><!-- put a recent version of opentelemetry sdk here --><version>
</dependency>
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-exporter-otlp-logs</artifactId>
  <version><!-- put a recent version of opentelemetry sdk here --><version>
</dependency>

STEP 2. Use the following code snippet to create and report a span:

SdkLoggerProvider loggerProvider =
      SdkLoggerProvider.builder()
        .addLogRecordProcessor(BatchLogRecordProcessor.builder(
          OtlpGrpcLogRecordExporter.builder()
            .setEndpoint("ingress.:443")
            .addHeader("Authorization", "Bearer <send-your-data-api-key>")
            .build()
        ).build())
        .setResource(Resource.create(Attributes.of(
          AttributeKey.stringKey("cx.application.name"), "my-test-application",
          AttributeKey.stringKey("cx.subsystem.name"), "my-test-subsystem")))
        .build();

    Logger logger = loggerProvider.loggerBuilder("test").build();

    logger.logRecordBuilder()
      .setSeverity(Severity.WARN)
      .setSeverityText("WARN")
      .setBody("Test log message")
      .emit();

    loggerProvider.forceFlush();

Limits & quotas

Coralogix places a hard limit of 10MB of data to our OpenTelemetry endpoints, with a recommendation of 2MB.

Limits apply to single requests, regardless of timespan.

Additional resources