## Overview

Using gRPC offers several advantages for developers building high-performance integrations. Because gRPC relies on Protocol Buffers (Protobuf) instead of JSON, requests and responses are transmitted in a compact binary format. This results in smaller payloads, faster serialization and deserialization, and lower CPU and memory overhead compared to traditional HTTP/JSON APIs. The result: reduced latency, higher throughput, and more efficient use of network resources.

## Requirements

To make gRPC API calls, you’ll need a tool like `grpcurl`. You can install [grpcurl](https://github.com/fullstorydev/grpcurl) or use any tool of your choice, but all of the examples below use `grpcurl`.

To use this API you need to create a [personal or team API key](https://coralogix.com/docs/user-guides/account-management/api-keys/api-keys/index.md). It’s recommended to use permission presets, as they are automatically updated with all relevant permissions. Alternatively, you can manually add individual permissions.

Any request that lacks proper authentication will be rejected. To authenticate, add an `Authorization Bearer <cx_api_key>` header to your API request. It contains a Bearer Token, which identifies a single user, bot user, or workspace-application relationship for authentication.

Select the https://api.\[[DOMAIN_VALUE]\]/api/v1/dataprime/query endpoint that corresponds to your Coralogix [domain](https://coralogix.com/docs/user-guides/account-management/account-settings/coralogix-domain/index.md) using the domain selector at the top of the page.

A typical gRPC API call is formatted like this: `grpcurl -H "Authorization: <cx_api_key>" -d "<data_json_object>" <host_name> <grpc_method>`

## Request payload body

When making a POST request, structure your payload using the format shown below. The query field is required; all other fields are optional and can be included in the metadata object to control execution parameters such as time range, query tier, or default source.

Note

Background queries use a different request structure (no `metadata` wrapper) and have their own required fields. See [Background queries gRPC API overview](#background-queries-grpc-api-overview) below.

The following example consists of a JSON structure that represents the request:

```json
{
  "query": "source logs | limit 100",
  "metadata": {                                    // metadata object is optional
    "tier": "TIER_ARCHIVE",                        // Search target of the query. Options: TIER_ARCHIVE, TIER_FREQUENT_SEARCH. Default: TIER_FREQUENT_SEARCH
    "syntax": "QUERY_SYNTAX_DATAPRIME",            // Query syntax. Options: QUERY_SYNTAX_DATAPRIME, QUERY_SYNTAX_LUCENE, others (see Swagger). Default: QUERY_SYNTAX_DATAPRIME
    "startDate": "2023-05-29T11:20:00.00Z",        // ISO 8601 date-time string defining the beginning of the query timeframe
    "endDate": "2023-05-29T11:30:00.00Z",          // ISO 8601 date-time string defining the ending of the query timeframe
    "defaultSource": "logs",                       // Default source used when source is omitted in a query
    "limit": 100,                                  // int32; limits the number of returned results. Max: 50,000 (TIER_ARCHIVE) or 12,000 (TIER_FREQUENT_SEARCH). Default: 2000
    "strictFieldsValidation": false,               // true = fail on unknown fields. Default: false
    "nowDate": "2023-05-29T12:00:00.00Z"           // substitutes for now() in query. Default: current time
  },
  "executionProfile": {                            // optional; controls the query engine's accuracy/performance trade-off
    "preset": "EXECUTION_PROFILE_PRESET_ACCURACY"  // EXECUTION_PROFILE_PRESET_ACCURACY (default): fresh results from source. EXECUTION_PROFILE_PRESET_PERFORMANCE: use caches for faster, approximate results
  }
}
```

For the full request and response schema reference (including all metadata fields, enum values, and limits), see the [DataPrime Query Service Swagger reference](https://coralogix.com/docs/static/swagger/index.md).

Note

`startDate`, `endDate`, and `nowDate` must be valid ISO 8601 date-time strings. If your local time zone is not UTC, either convert the timestamp to UTC or include the time zone offset (e.g., `2023-05-29T11:20:00.00-07:00` for Pacific Daylight Time, `2023-05-29T11:20:00.00+05:30` for India Standard Time).

Note

When querying **archived** logs, you must explicitly specify `"tier": "TIER_ARCHIVE"` in the metadata object. Otherwise, the default is `"tier": "TIER_FREQUENT_SEARCH"`.

Note

While the examples above are labeled as JSON for readability, they are not true JSON payloads. When using tools like `grpcurl`, these structures are parsed and converted into Protobuf's binary format under the hood. The JSON-like syntax is a convenience for human readability and documentation purposes only.

Field names in the JSON examples use the canonical camelCase form (matching the [Swagger reference](https://coralogix.com/docs/static/swagger/index.md)); the Protobuf message definitions use the underlying snake_case field names. `grpcurl` accepts either form.

## Direct API query

### `com.coralogixapis.dataprime.v1.DataprimeQueryService/Query`

```protobuf
message Query {
    string query = 1;
}
```

```json
{
  "query": "source logs | limit 10"
}
```

Note

Protobuf key-value pairs are less descriptive than those in JSON documents because the keys are represented by numeric tags instead of human-readable names.

### Make the API call

```bash
grpcurl -H "Authorization: <cx_api_key>" \
    -d '{ "query": "source logs | limit 10", "metadata": { "tier": "TIER_ARCHIVE", "syntax": "QUERY_SYNTAX_DATAPRIME" }}' \
    api.[[DOMAIN_VALUE]]:443 \
    com.coralogixapis.dataprime.v1.DataprimeQueryService/Query
```

### Response

gRPC responses are streamed as binary data and may arrive in multiple messages instead of a single response object, as in REST. For background queries, the server sends the `queryId` first to acknowledge receipt, followed by a separate message containing the actual `result` data. These parts are logically connected but delivered in sequence over the stream. In most tools, such as grpcurl, these streamed messages are printed as separate JSON blocks.

For the full response schema (including all `result` and `error` field shapes), see the [`QueryResponse` reference](https://coralogix.com/docs/static/swagger/#schemacom.coralogixapis.dataprime.v1.queryresponse).

```json
{
  "queryId": {
    "queryId": "a1e02902-9ecf-490d-8b3f-ce9dba179e04"
  }
}
{
  "result": {
    "results": [
      {
        "metadata": [
          ...
        ],
        "labels": [
          ...
        ],
        "userData": ...
      },
      ...
    ]
  }
}
{
  "statistics": {
    "status": "COMPLETED",
    "e2eDurationMs": "260",
    "outputRowCount": "1",
    "storage": {
      "objectStore": {
        "bytesRead": "0",
        "crossRegionBytesRead": "0",
        "limits": {
          "scan": { "reached": false, "limit": "50000000000", "measuredValue": "454108" },
          "shuffleSize": { "reached": false, "limit": "1000000000", "measuredValue": "704" },
          "filesRead": { "reached": false, "measuredValue": "0" },
          "column": { "reached": false }
        }
      }
    }
  }
}
```

Every direct query response ends with a `statistics` object describing execution status, duration, row count, bytes read, and per-resource limit signals. Integer values are returned as JSON strings to preserve 64-bit precision. For the full field reference, see [Query statistics](https://coralogix.com/docs/dataprime/API/direct-archive-query-http/#query-statistics) on the HTTP API page. The same `statistics` model is returned by [`GetBackgroundQueryStatus`](#getbackgroundquerystatus) once a background query reaches a terminated state.

## Background queries gRPC API overview

The Background queries gRPC API enables you to run high-latency, long-running queries asynchronously using DataPrime or Lucene syntax from scripts or the CLI. Designed for recurring, extensive analytical tasks—such as monthly or quarterly reports—this API allows you to offload complex queries to run in the background, freeing you to continue real-time monitoring within the Coralogix platform.

Use it to:

- Seamlessly query archived logs and spans, regardless of time range.
- Leverage higher scan, file, and shuffle limits for deeper data access.
- Take advantage of extended execution times for heavy analytical workloads.

The API supports the following background gRPCs:

- [SubmitBackgroundQuery](#submitbackgroundquery)
- [GetBackgroundQueryStatus](#getbackgroundquerystatus)
- [CancelBackgroundQuery](#cancelbackgroundquery)
- [GetBackgroundQueryData](#getbackgroundquerydata)

Note

The structure of background query requests is different from that of direct query requests — background query requests don't contain the `metadata` nested object; the relevant fields (`syntax`, `startDate`, `endDate`, `nowDate`) are top-level.

## SubmitBackgroundQuery

### `com.coralogixapis.dataprime.v1.DataprimeQueryService/SubmitBackgroundQuery`

```protobuf
message SubmitBackgroundQueryRequest {
  google.protobuf.StringValue query = 1;
  com.coralogixapis.dataprime.v1.QuerySyntax syntax = 2;
  // Start of query time range (inclusive)
  optional google.protobuf.Timestamp start_date = 3;
  // End of query time range (exclusive)
  optional google.protobuf.Timestamp end_date = 4;
  // Used by functions like now() in DataPrime
  optional google.protobuf.Timestamp now_date = 5;
  // Where to write results. When omitted, writes to an engine-owned temporary
  // dataset (system/engine.resultsets.bg_queries.{unique_id}) with 30-day retention.
  optional ResultsDestination results_destination = 6;
  // Fully qualified name (FQN) of the default source when omitted in the query. Defaults to "default/logs".
  optional string default_source = 7;
  // Optional. Starts with up to two underscores followed by a letter, then letters,
  // numbers, underscores, or dots; no consecutive or trailing dots; 2-255 characters.
  // Defaults to an auto-generated name.
  optional string name = 8;
  // Optional description, up to 2048 characters. Defaults to empty string.
  optional string description = 9;
}
```

```json
{
  "query": "limit 3", 
  "syntax":"QUERY_SYNTAX_DATAPRIME", 
  "startDate": "2024-01-20T00:00:00Z", 
  "endDate": "2024-01-20T01:00:00Z", 
  "nowDate":"2024-01-20T02:00:00Z"
  }
```

### Make the API call

```bash
 grpcurl -H "Authorization: <cx_api_key>" \
    -d '{ "query": "limit 3", "syntax":"QUERY_SYNTAX_DATAPRIME", "startDate": "2024-01-20T00:00:00Z", "endDate": "2024-01-20T01:00:00Z", "nowDate":"2024-01-20T02:00:00Z"}' \
    api.[[DOMAIN_VALUE]]:443 \
    com.coralogixapis.dataprime.v1.DataprimeQueryService/SubmitBackgroundQuery
```

### Response

The response object contains a `queryId`. This ID is used to do follow up operations on the background query.

```json
{
  "queryId": "1712b991-b9be-455f-944e-308f6aa6bbbe"
}
```

## Writing results to a dataset

To write results to a persistent, user-defined dataset instead of the temporary default, set `resultsDestination`. The target dataset must already exist under the `default` dataspace (see [Background Queries](https://coralogix.com/docs/user-guides/dataengine/background_queries_v2/index.md) for the in-app equivalent and [Dataspace Management](https://coralogix.com/docs/user-guides/data-layer/default-dataspace/user-defined-datasets/#create-a-dataset) to create one):

```bash
grpcurl -H "Authorization: <cx_api_key>" \
    -d '{
      "query": "source logs | limit 1000",
      "syntax": "QUERY_SYNTAX_DATAPRIME",
      "startDate": "2024-01-20T00:00:00Z",
      "endDate": "2024-01-20T01:00:00Z",
      "resultsDestination": {
        "persistentDestination": {
          "datasets": [
            { "dataspace": "default", "dataset": "my_summary_dataset", "writeMode": "DATASET_WRITE_MODE_APPEND" }
          ],
          "partitioning": { "dateHourPartitioned": { "dataprimePath": "$m.timestamp" } }
        }
      }
    }' \
    api.[[DOMAIN_VALUE]]:443 \
    com.coralogixapis.dataprime.v1.DataprimeQueryService/SubmitBackgroundQuery
```

The destination fields use the following message types.

### ResultsDestination

Determines where the query writes its results. Set **at most one** of `temporaryDestination` or `persistentDestination`. When `resultsDestination` is omitted, results go to the temporary destination.

```protobuf
message ResultsDestination {
  oneof kind {
    // Results stored in the engine-owned system/engine.resultsets.bg_queries.{unique_id} dataset for 30 days.
    TemporaryDestination temporary_destination = 1;
    // Results written to a user-defined dataset in the caller's team.
    PersistentDestination persistent_destination = 2;
  }

  message TemporaryDestination {}

  message PersistentDestination {
    // v1 accepts at most one entry. Submitting more than one returns a BAD_REQUEST error.
    repeated DatasetDestination datasets = 1;
    // When omitted, uses the same partitioning as the query's results.
    optional Partitioning partitioning = 2;
  }
}
```

### DatasetDestination

```protobuf
message DatasetDestination {
  // The dataset must already exist; this API does not create datasets, and it must be under the `default` dataspace.
  google.protobuf.StringValue dataspace = 1;
  google.protobuf.StringValue dataset = 2;
  DatasetWriteMode write_mode = 3;

  enum DatasetWriteMode {
    DATASET_WRITE_MODE_UNSPECIFIED = 0;
    DATASET_WRITE_MODE_APPEND = 1;
    DATASET_WRITE_MODE_OVERWRITE = 2;
  }
}
```

### Partitioning

```protobuf
message Partitioning {
  oneof kind {
    Unpartitioned unpartitioned = 1;
    DateHourPartitioned date_hour_partitioned = 2;
  }

  // Partition results on values of timestamp type with date/hour granularity.
  message DateHourPartitioned {
    // DataPrime keypath of a timestamp to partition on, e.g. `$m.timestamp`.
    // The keypath must exist in the query's results and be of timestamp type.
    string dataprime_path = 1;
  }

  message Unpartitioned {}
}
```

## GetBackgroundQueryStatus

### `com.coralogixapis.dataprime.v1.DataprimeQueryService/GetBackgroundQueryStatus`

```protobuf
message GetBackgroundQueryStatusRequest {
  // Generated query ID that can be later used to obtain status and results
  string query_id = 1;
}
```

```json
{ 
  "queryId": "412036c3-04be-431a-b1e2-9ebf971be6c6" 
}
```

### Make the API call

```bash
grpcurl -H "Authorization: <cx_api_key>" \
    -d '{ "queryId": "412036c3-04be-431a-b1e2-9ebf971be6c6" }' \
    api.[[DOMAIN_VALUE]]:443 \
    com.coralogixapis.dataprime.v1.DataprimeQueryService/GetBackgroundQueryStatus
```

### Response

The response is a `GetBackgroundQueryStatusResponse` whose `status` oneof is one of `running`, `terminated`, or `waitingForExecution`. A terminated query carries its own `success`, `error`, or `cancelled` outcome:

```protobuf
message GetBackgroundQueryStatusResponse {
  // Status of the query
  oneof status {
    Running running = 1;
    Terminated terminated = 2;
    WaitingForExecution waiting_for_execution = 3;
  }

  // Returned when the query is waiting for execution.
  message WaitingForExecution {}

  // Returned when the query is running.
  message Running {
    // Time at which the query started running.
    google.protobuf.Timestamp running_since = 1;
  }

  // Returned when the query has terminated.
  message Terminated {
    google.protobuf.Timestamp running_since = 2;
    google.protobuf.Timestamp terminated_at = 3;
    oneof status {
      Success success = 4;
      Error error = 5;
      Cancelled cancelled = 6;
    }

    message Success {}
    message Cancelled {}

    message Error {
      oneof error {
        TimedOut timed_out = 1;
        Cancelled cancelled = 2;
        Failed failed = 3;
      }

      message TimedOut {}
      message Cancelled {}
      message Failed {
        google.protobuf.StringValue reason = 1;
      }
    }
  }
}
```

**Example** (terminated):

```json
{
  "terminated": {
    "runningSince": "2025-01-21T09:35:01Z",
    "terminatedAt": "2025-01-21T09:35:01Z",
    "success": {}
  },
  "submittedAt": "2025-01-21T09:35:00Z",
  "metadata": [
    {
      "statistics": {
        "bytesScanned": "506070"
      }
    }
  ]
}
```

## CancelBackgroundQuery

### `com.coralogixapis.dataprime.v1.DataprimeQueryService/CancelBackgroundQuery`

```protobuf
message CancelBackgroundQueryRequest {
  // Generated query ID that can be later used to obtain status and results
  string query_id = 1;
}
```

```json
{
  "queryId": "412036c3-04be-431a-b1e2-9ebf971be6c6"
}
```

### Make the API call

```bash
grpcurl -H "Authorization: <cx_api_key>" \
    -d '{ "queryId": "412036c3-04be-431a-b1e2-9ebf971be6c6" }' \
    api.[[DOMAIN_VALUE]]:443 \
    com.coralogixapis.dataprime.v1.DataprimeQueryService/CancelBackgroundQuery
```

### Response

```json
{}
```

## GetBackgroundQueryData

### `com.coralogixapis.dataprime.v1.DataprimeQueryService/GetBackgroundQueryData`

```protobuf
message GetBackgroundQueryDataRequest {
  // Generated query ID that can be later used to obtain status and results
  string query_id = 1;
}
```

```json
{
  "queryId": "412036c3-04be-431a-b1e2-9ebf971be6c6"
}
```

### Make the API call

```bash
grpcurl -H "Authorization: <cx_api_key>" \
    -d '{ "queryId": "412036c3-04be-431a-b1e2-9ebf971be6c6" }' \
    api.[[DOMAIN_VALUE]]:443 \
    com.coralogixapis.dataprime.v1.DataprimeQueryService/GetBackgroundQueryData
```

### Response

```json
{
  "queryId": {
    "queryId": "a1e02902-9ecf-490d-8b3f-ce9dba179e04" // initial acknowledgment
  }
}
{
  "result": {  // returned later via streaming
    "results": [
      {
        "metadata": [
          ...
        ],
        "labels": [
          ...
        ],
        "userData": ...
      },
      ...
    ]
  }
}
```

## Response handling

gRPC APIs use standard [gRPC status codes](https://grpc.io/docs/guides/status-codes/) instead of traditional HTTP response codes. When a request fails, the response will include one of the following codes, along with a descriptive error message.

| **Status code**     | **Description**                                                                 |
| ------------------- | ------------------------------------------------------------------------------- |
| `OK`                | The request was successful.                                                     |
| `INVALID_ARGUMENT`  | The request was malformed — e.g., a required field was missing or misformatted. |
| `UNAUTHENTICATED`   | The API key is missing or invalid.                                              |
| `PERMISSION_DENIED` | The API key is valid but lacks the required permissions.                        |
| `NOT_FOUND`         | The specified `queryId` does not exist or has expired.                          |

A response with `INVALID_ARGUMENT` usually indicates that the request body doesn't follow the expected schema or contains invalid enum values or date formats.

If you receive `UNAUTHENTICATED` or `PERMISSION_DENIED`, check your [API keys page](https://coralogix.com/docs/user-guides/account-management/api-keys/api-keys/index.md) or contact your workspace administrator to ensure you have the correct credentials and permissions.

## Limitations

When a limit is breached, a **warning message** is displayed.

See the limitations page for a comprehensive list of [direct query limitations](https://coralogix.com/docs/dataprime/API/limitations/#direct-query-limitations) and [background query limitations](https://coralogix.com/docs/dataprime/API/limitations/#background-query-limitations).

## Additional resources

|                     |                                                                                             |
| ------------------- | ------------------------------------------------------------------------------------------- |
| Coralogix Endpoints | [Coralogix Endpoints](https://coralogix.com/docs/integrations/coralogix-endpoints/index.md) |