# Expressions ## Language constructs This section provides an overview of the language constructs supported by DataPrime. These constructs serve as the building blocks for complex query expressions, combining operations to communicate what they want to achieve or calculate. The constructs are: - **Validated** during query writing. - **Executed** at runtime. - **Runtime failures** are returned as `null`. - In **boolean predicates**, `null` values are treated as `false`. These constructs are effectively expressions as well, and are the building blocks which allow users to “compose” more complex expressions. Let’s discuss this separately. ### Literals Literals are constant values that can represent different data types. They are the most basic form of data and are used to express fixed values directly in the query. Examples: - **Boolean:** `true`, `false` - **String:** `"example"`, `'single-quoted string'` - **Number:** `123`, `45.67` - **Array:** `[1, 2, 3]`, `["one", "two", "three"]` - **Null**: `null` Example literals for all data types can be found [here](https://coralogix.com/docs/dataprime/beginners-guide/data-types/index.md). ### Nested field access Fields inside objects can be accessed using dot-notation or map-notation. This is useful when working with [nested data](https://coralogix.com/docs/dataprime/beginners-guide/how-data-is-represented-in-dataprime/index.md). Examples: - **Dot Notation:** `user.name` (to access the `name` field inside `user`) - **Map Notation:** `$d['my-special.log-key']` (for fields with special characters in their keys) ### Math operations Basic mathematical operations allow you to perform arithmetic calculations. Supported operations: | **Operation** | **Symbol** | | -------------- | ---------- | | Addition | `+` | | Subtraction | `-` | | Multiplication | `*` | | Division | `/` | | Modulus | `%` | Example: ```dataprime 5 + 10 * 2 // Result: 25 ``` ### Boolean operations Boolean operations are used to combine or negate boolean expressions. Supported operations: | **Operation** | **Symbol** | | ------------- | ---------- | | AND | `&&` | | OR | \` | | NOT | `!` | Example: ```dataprime true && false // Result: false ``` ### Comparison operations Comparison operations allow you to compare two values and return a boolean result. Supported operations: | Operation | Symbol | | ---------------------- | ------ | | Equals | `==` | | Not Equals | `!=` | | Greater Than | `>` | | Less Than | `<` | | Greater Than or Equals | `>=` | | Less Than or Equals | `<=` | Examples: ```dataprime x > 10 && x <= 20 // Check if x is larger than 10 and equal to or less than 20 ``` ```dataprime source logs | filter response_time > 500 && status_code == 200 //Filters logs where the response time is greater than 500 milliseconds and the status code is 200 ``` ### Text search Text search operations are used to match patterns within strings. Supported constructs: - **Text Match:** `~` (matches a pattern) - **Fuzzy Match:** `~~` (wildfind which works on $d) Example: ```dataprime source logs | filter name ~ 'emailservice-server' ``` ### String concatenation String concatenation is the joining of two or more string fields, such that they produce a single string that is made up of each original string. Concatenation is done using the `concat` function. For example, consider the following document: ```dataprime { "stringa": "Hello ", "stringb": "World!" } ``` We can *concatenate* these two values, like so: ```dataprime create stringc from stringa.concat(stringb) ``` Resulting in the following documents: ```dataprime { "stringa": "Hello ", "stringb": "World!", "stringc": "Hello World!" } ``` ### String interpolation String interpolation allows dynamic value concatenation within strings, improving readability. Syntax: - A string is wrapped in backticks (`` ` ``). - Inside the string, expressions are enclosed in `{}`. Examples: ```dataprime `The value of x is {x}` // Inserts the value of x into the string `Alert {alert_name} has been triggered at {timestamp}` `This user has an ID of: {user_id}` ``` If you want to generate a custom message that includes a user ID from your logs, you can do the following: ```dataprime source logs | filter user_id != null | replace custom_message with `User ID: {user_id} logged in.` ``` This query filters logs to include only those with a user ID, then creates a custom message incorporating that user ID. ### Time-based operations Time constructs allow you to work with timestamps and intervals. You can represent, manipulate, and compare time-related data. Supported formats: - **Timestamp Literal:** `@`, `@`, `@()` - **Interval Literal:** `1h`, `30m`, `2d` Examples: - **Time Bucket:** `timestamp / 1h` (to group timestamps by hour) - **Time Arithmetic:** `@2021-01-01T12:00:00 - 1h` (to subtract one hour from a timestamp) See **Working with time in DataPrime** for a full review of time-based operations. ### Data type casting Casting allows you to convert one data type into another. This is useful when you need to manipulate or transform data into a different form. Syntax: ```dataprime : ``` Example: ```dataprime 123 :string // Converts the number 123 into a string "123" ``` ### Comments Comments can be added to queries to explain the logic or temporarily disable code. Supported comment styles: - **Single-line comment:** `//` - **Multi-line comment:** `/* ... */` Example: ```dataprime // This is a single-line comment /* This is a multi-line comment */ ``` ### Free/Wild text search In DataPrime, there are two types of text searches: **free-text search** and **wild-text search**. - **Free-text search** refers to searching within a specific key or field in the event data. - **Wild-text search** refers to searching across the entire event data, not restricted to a particular field. **Free-text search** - **Syntax:** ` ~ ` - This type of search allows you to filter by a specific key, similar to Lucene's `key:text` construct. - It can be used as part of filter conditions in queries to match a string within a specific field. **Wild-text search** - **Syntax:** `$d ~~ ` - This search is applied across all fields of the event data, not just one specific field. It behaves similarly to Lucene’s `` construct, allowing for broader searches. - Like free-text, it can also be used within filters. **Lucene operator** - DataPrime supports a `lucene` operator that can take a full Lucene query as a string, enabling complex text searches using Lucene’s syntax directly. **Query optimization** - It's recommended to place text searches early in the query to optimize performance. - The system has been optimized to the point where, in some cases, **wild-text search** may be faster than **free-text search** for broader queries.