Toxicity

The toxicity detection guardrail protects your LLM applications from generating or processing toxic, harmful, or offensive content such as hate speech, threats, harassment, and abusive language.

What you need

Python 3.10 or higher.
cx-guardrails installed. See Getting Started with Guardrails.
A Team API key with the AiObservability role preset, used as CX_GUARDRAILS_TOKEN. The AiObservability preset includes AI-GUARDRAILS:MANAGE and all other permissions required to use Guardrails.
Environment variables configured: CX_GUARDRAILS_TOKEN, CX_GUARDRAILS_ENDPOINT.
The AI-GUARDRAILS:MANAGE permission.

Install the SDK

pip install cx-guardrails

Set up environment variables

export CX_GUARDRAILS_TOKEN="your-coralogix-guardrails-api-key"
export CX_GUARDRAILS_ENDPOINT="https://api.<domain>.coralogix.com/api/v1/guardrails/guard"
export CX_TOKEN="your-coralogix-send-your-data-key"
export CX_ENDPOINT="https://your-domain.coralogix.com"

# Optional: Application metadata for observability
export CX_APPLICATION_NAME="my-app"
export CX_SUBSYSTEM_NAME="my-subsystem"

Set up observability

To send guardrail spans to AI Center, set up OpenTelemetry trace export. For the full overview, see OpenTelemetry integration for AI Center.

Install the OpenTelemetry packages:

pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-grpc

Export the OTLP environment variables:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingress.:443"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <your-api-key>"
export OTEL_SERVICE_NAME="my-ai-service"
export OTEL_RESOURCE_ATTRIBUTES="cx.application.name=my-app,cx.subsystem.name=my-subsystem"
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true
export OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental

Initialize the tracer provider in your application before any guardrail or LLM calls:

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor


def configure_otel() -> TracerProvider:
    resource = Resource.create()
    provider = TracerProvider(resource=resource)
    provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
    trace.set_tracer_provider(provider)
    return provider

Usage

import asyncio
from cx_guardrails import Guardrails, Toxicity, GuardrailsTriggered

async def main():
    guardrails = Guardrails()
    async with guardrails.guarded_session():
        try:
            await guardrails.guard_prompt(
                prompt="Hello, how can I help you today?",
                guardrails=[Toxicity()],
            )
            print("No toxicity detected")
        except GuardrailsTriggered as e:
            print(f"Toxicity detected: {e}")

asyncio.run(main())

Configuration options

Custom threshold

Adjust detection sensitivity (0.0 to 1.0, default 0.7):

# Lower threshold — more sensitive
await guardrails.guard_prompt(
    prompt=user_input,
    guardrails=[Toxicity(threshold=0.5)],
)

# Higher threshold — less sensitive
await guardrails.guard_prompt(
    prompt=user_input,
    guardrails=[Toxicity(threshold=0.9)],
)

Threshold: Defines the value from which a guardrail action is triggered. When the threshold is met or exceeded, the guardrail action is executed, returned through the API, and the system marks the event as an issue.

Next steps

Define domain-specific guardrail policies using natural language with Custom guardrail policies.

Need help? Contact Support.

What's new? Find out here.

LLM? Read llms.txt.

Previous PII

Next Custom guardrail policies