TCO Optimizer

The Total Cost of Ownership (TCO) Optimizer routes logs and traces to the right priority based on their business value. You define policies that match incoming data and route it to one of four outcomes:

• High (Frequent Search) — optionally stored in Coralogix hot storage (OpenSearch) for fast queries
• Medium (Monitoring) — stored in customer-owned S3, queryable via DataPrime, with alerting and dashboarding
• Low (Compliance) — stored in customer-owned S3, instantly queryable via DataPrime for compliance and long-term retention
• Blocked — discarded at ingestion

Policies are evaluated in order. The first matching policy applies, and no other policies are evaluated.

Prerequisites

To create or edit TCO policies, you need:

• Admin or appropriate role permissions in your Coralogix account
• Access to Data Flow in the Coralogix UI

How TCO routing works

When a log or trace is ingested, Coralogix evaluates your enabled policies from top to bottom:

1. Evaluate the first policy.
2. If the event matches, route it to the policy's configured priority and stop.
3. If it does not match, evaluate the next policy.
4. If no policy matches, route the event to High (Frequent Search).

Order matters. Place narrow exception policies above broader catch-all policies.

TCO priority levels explained

High (Frequent Search)

(Optional) Data is retained in Coralogix hot storage (OpenSearch). This priority is designed for business-critical data that requires fast queries, real-time alerting, and immediate investigation. High (Frequent Search) is the default priority for data that does not match any policy.

Medium (Monitoring)

Raw data is retained in customer-owned storage and remains instantly queryable via DataPrime, with added support for alerting and dashboarding. Use this priority for monitoring and ongoing analysis at scale.

Low (Compliance)

Raw data is retained in customer-owned storage and remains instantly queryable via DataPrime. Use this priority for compliance requirements and long-term retention where frequent querying is not required.

Blocked

Data is dropped at ingestion and not stored. Blocked data incurs ingestion processing cost only — no storage or retention charges. Use Blocked to eliminate storage costs for unwanted data.

Features by priority level

Each priority level provides different capabilities across the platform:

Feature	High (Frequent Search)	Medium (Monitoring)	Low (Compliance)
AI Evaluators	✅	✅
Alerts	✅	✅
APM with Events2Metrics	✅	✅
APM with Span Metrics	✅	✅	✅
Archive	✅	✅	✅
Background Queries	✅	✅	✅
Custom Dashboards	✅	✅
Data Enrichment	✅	✅
Events2Metrics	✅	✅
Explore	✅	✅	✅
Lightning Queries	✅
LiveTail	✅	✅	✅
Log Parsing	✅	✅	✅
Loggregation	✅	✅
Real User Monitoring	✅	✅
Session Replay	✅	✅	✅

Blocked data is discarded at ingestion — no features are available.

Policies and filters

A TCO policy is defined by:

• A filter that selects which logs or traces the policy applies to
• A priority outcome: High (Frequent Search), Medium (Monitoring), Low (Compliance), or Blocked
• An archive retention setting

Filters are defined using DataPrime Expression Language (DPXL). DPXL lets you match on:

• Standard labels such as application and subsystem
• Metadata such as severity and timestamp
• Any fields in the event body, including nested fields and custom keys

For common use cases, the policy editor provides a Builder that generates DPXL automatically. You can also edit the policy filter directly as a DPXL expression.

Evaluation order matters. Policies are evaluated from top to bottom. The first matching policy applies and no subsequent policies are evaluated. Data that does not match any policy goes to High (Frequent Search) by default. Reorder policies in the UI via drag-and-drop.

DPXL in TCO

TCO policy filters use the DataPrime Expression Language (DPXL) to match incoming events. DPXL supports flexible boolean logic, string functions, and set membership checks across labels, metadata, and user data fields.

Expression format

In the UI, the version prefix is handled automatically. When using the API, prefix expressions with <v1>:

<v1> <boolean-expression>

Keypaths in TCO policies

DPXL keypaths are grouped into three categories:

• $m — metadata fields
• $l — label fields
• $d — user data fields (default when no prefix is specified)

Common keypaths for TCO policies:

• $l.applicationname — application name
• $l.subsystemname — subsystem name
• $m.severity — severity level

You can also reference any user data field, for example:

• $d.environment
• $d.tenant_id
• $d.kubernetes.pod_name

Operators and functions

DPXL supports boolean logic and comparison operators:

• &&, ||, !
• ==, !=, >, >=, <, <=

String functions:

• .startsWith()
• .endsWith()
• .contains()
• .matches()

Set membership:

• .in(...)

DPXL examples

Match an application:

<v1> $l.applicationname == 'payments-service'

Match application and subsystem:

<v1> $l.applicationname == 'payments-service' && $l.subsystemname == 'worker'

Match by severity:

<v1> $m.severity.in(ERROR, CRITICAL)

Match by event data:

<v1> $d.environment == 'AWS' && $d.content == 'payment'

Combine labels and user fields:

<v1> $l.applicationname.startsWith('prod-') && $d.tenant_id:string.in('t1','t2','t3')

Type casting: DPXL infers types when possible. When it cannot — notably with .in(...) — cast the keypath explicitly:

<v1> !$d.country:string.in('us','il','gr')

DPXL limits

• Maximum number of expressions: 150
• Maximum depth: 20
• .in(...) may require explicit type casting when DPXL cannot infer the keypath type

Create a policy

Basic setup

1. Navigate to Data Flow, then TCO Optimizer.
2. Select Logs or Traces.
3. Select + New policy.
4. In Details, set:
   • Policy name (required)
   • Description (optional)
   • Policy order: First places the policy at the top of the evaluation list — it is evaluated before all existing policies. Last places it at the bottom. Policies are evaluated in order from top to bottom; when a log or trace matches a policy, the system stops and does not evaluate any subsequent policies.
5. In Filters, define the policy filter:
   • Use the Builder to add conditions for application, subsystem, and severity using operators is, is not, includes, starts with, or
   • Select Expression and enter a DPXL filter directly
6. In Priority, choose the routing outcome: High (Frequent Search), Medium (Monitoring), Low (Compliance), or Blocked.

As you select a priority, the card displays the estimated Units (U) and the expected change compared to your baseline.

Archive retention

Archive retention assigns a label as a tag on the Parquet files written to your S3 bucket. Use these tags in your cloud storage lifecycle policies to control how long archived data is retained. Coralogix does not manage deletion — retention duration is controlled by your storage provider.

1. In Archive retention, choose a label: None, Default, Short, Intermediate, or Long.
2. Enable the policy to take effect immediately.
3. Select Create.

Edit, disable, or delete a policy

• To edit a policy, select the policy row and modify its filter, priority, or archive retention.
• To disable a policy, toggle the policy off. Disabled policies are skipped during evaluation.
• To reorder policies, drag the policy to a new position in the list.
• To delete a policy, select the policy and choose Delete.

Changes take effect immediately. If a policy causes unexpected routing, disable it to revert to the previous behavior.

Policy order and exception handling

Because the first matching policy applies, use this pattern:

• Put specific exceptions first.
• Put broad defaults last.

Example

Broad policy: all logs from payments-service go to Medium (Monitoring).

Exception: logs from payments-service with ERROR or CRITICAL severity go to High (Frequent Search).

Exception policy (place first):

<v1> $l.applicationname == 'payments-service' && $m.severity.in(ERROR, CRITICAL)

Broad policy (place below):

<v1> $l.applicationname == 'payments-service'

Usage overview

Use the Usage overview to understand how your daily quota is consumed across priorities and how routing affects cost.

Hover over any priority in the top bar to open the Usage overview panel.

The panel includes:

Daily team quota

Shows your available daily unit quota. Coralogix uses a unit-based pricing model, where a unit is a universal billing metric per observability pillar.

Yearly savings

Shows the estimated percentage saved by routing data to Medium (Monitoring) and Low (Compliance) instead of High (Frequent Search).

Usage by priority

Shows how ingested data is distributed across priorities. For each priority, you can see how many units are consumed by each data type and how this contributes to your total daily quota.

Units per pillar

Shows unit consumption per observability pillar, broken down by priority.

Use the Usage overview to quickly assess quota consumption, validate the impact of your TCO policies, and identify opportunities for further cost optimization.

Statistics

Statistics show ingested logs or traces organized by key dimensions and their contribution to usage. Use this view to identify high-volume data and turn it into policies.

Use this view to:

• Group by application, subsystem, severity, affecting policy, and priority
• Expand an application to see its subsystems, then expand a subsystem to see its severity breakdown
• See contribution to % Quota, Data sent, and Units (U) for each combination
• Validate expected impact before saving

Row menu actions

• Add rule — Pre-fills the policy form with the row's values
• Drill into logs — Opens a query filtered to the row's values
• Copy as filter — Copies application, subsystem, and severity chips for reuse

Slice and filter the data

• Group by chips — Choose which fields to include in the breakdown. Drag and drop chips to rearrange them.
• Search — Filter rows by name (supports partial matches).
• Include/exclude values — Use chip menus to add or remove specific values.
• Reset — Clear temporary filters and return to the default settings.

Create a policy from statistics

Turn any selection into a policy without leaving the page:

1. Select a row.
2. Choose Add rule. The Create policy form opens with the Builder pre-filled.
3. Choose a priority: High (Frequent Search), Medium (Monitoring), Low (Compliance), or Blocked.
4. (Optional) Set archive retention.
5. Review the Preview panel for data sent, % quota, and units impact.
6. Toggle Enable policy on to activate immediately.
7. Select Create.

The new policy appears in the TCO rules table. Order matters — set Policy order to place it First (highest precedence) or Last.

Understand the "affecting policy" column

When a combination already matches a policy, the Affecting policy column displays the policy currently in effect.

Use it to:

• Avoid creating duplicate policies
• Spot combinations that fall into the default High (Frequent Search) priority (shown as — or none)
• Decide whether to edit an existing policy or add a higher-precedence one

API and Terraform

TCO policies can also be managed via the Coralogix API or Terraform provider. See the TCO Optimizer HTTP API and Terraform documentation for details.

Log overrides: End-of-life and migration

The Override table in your TCO UI is slated for deprecation on March 31, 2026. From that point forward, all existing override rules will automatically appear as policies at the top of the Log policies panel. As granular policies, they will be executed before more general policies, preserving the current override functionality.

Before March 31, 2026, during the hybrid phase, the Override panel will continue to exist, and override APIs will remain functional.

How can I prepare?

UI users are strongly encouraged to migrate their overrides to granular log policies before that time. To do so, hover over an override, then click on the convert icon.

• The rule is removed from the override table.
• A new TCO policy is created with the same conditions and settings.
• The converted rule appears in the Log policies table and follows its execution order.
• This action can't be undone.

Should I delete or convert an existing override?

Compare the override TCO priority with the fallback priority. If the override priority and fallback priority are identical, the override may be deleted.

Find out more in the Overrides EoL Announcement.