Compliance Guide

Base-Level Enablement Documentation: Companion to the AI CTRL Administrator Guide

This guide covers the standard workflow for accessing and using the AI CTRL Compliance Platform, built on Elastic / Kibana (version 8.19). It is intended for administrators and compliance stakeholders responsible for usage governance and auditing, and assumes you have read the AI CTRL Administrator Guide.

Accessing the Platform

Your organization is provisioned with a dedicated, single-tenant Elastic deployment. Access is controlled through OneLogin, the identity provider used specifically for authentication into this compliance platform. This is a separate system from the AI CTRL gateway itself.

1. Account invitation: you will receive an invitation from no-reply@onelogin.com. This may be flagged as spam or junk mail: check those folders if it does not arrive.
2. Follow the invite link to: set your password, and register multi-factor authentication (MFA) using the OneLogin Protect mobile app.
3. Locate your app tile: once logged into your OneLogin tenant, find the app tile labeled "Elastic AI [Organization] Admin" (the naming reflects your organization). This may appear on your default dashboard or under a secondary "all apps" tab if not frequently used.
4. Log in: click the app tile, then click Expedient Client Login to land on the Elastic home screen.
5. Password reset: use Forgot Password on the OneLogin login screen if needed. Your registered MFA remains active: you will simply re-enter a verification code after resetting.

A separate OneLogin app tile may exist for other, non-AI services your organization uses. Make sure you select the tile specifically labeled for the AI / Elastic compliance platform.

Platform Data Structure (Baseline Concepts)

All AI gateway activity is stored under the Elastic data view named prompt. Always confirm you are on this data view: the default logs data view contains unrelated backend system data and will not give you a meaningful picture of usage.

Tokens

Tokens are the unit LLMs use to process content, at an approximate ratio of 3 words to 4 tokens.

• Prompt tokens: the user's input.
• Response tokens: the AI's output.
• Total tokens: prompt plus response.

Because each new message in an ongoing chat carries prior conversation history forward as context, prompt tokens tend to grow the longer a conversation continues, and grow further still if data files are attached, since their extracted text is injected into the prompt. As a result, the ratio of prompt tokens to response tokens in a given chat is typically well above 1:1, and this is expected behavior rather than a sign of a problem.

Filtering for Accurate Usage Data

Two categories of activity should be filtered out to see what your actual users are doing, separate from platform overhead:

• Admin / service account activity used by your delivery team for demonstration, troubleshooting, or configuration. This does not count against your billing, credits, or user seats. Filter value: admin@ai.expedient.cloud.
• Text-embedding model activity such as a text-extraction model triggered automatically when a user uploads a file. This does not count against token or credit consumption. Filter value: text-embedding-3-small.

Entries in the compliance platform cannot be deleted by end users, even if the corresponding chat is deleted on the front end. This supports long-term audit and governance requirements.

Dashboard

Location: Menu → Analytics → Dashboards

The dashboard provides an at-a-glance operational view of usage. It is not intended for deep-dive analysis; use Discover for that (see the next section).

Steps

1. Navigate to Analytics → Dashboards and open the pre-built AI Gateway Monitoring dashboard.
2. Set the time range using the calendar/date picker. The default may show only the last 15 minutes, so switch to a preset range (such as Last 7 Days) or a custom absolute/relative range.
3. Apply filters to exclude the admin/service account (by username) and the text-embedding/extraction model, so the view reflects only genuine user activity.

Panels

Customizing the Dashboard

Panels can be edited, duplicated, or reconfigured using the built-in Kibana dashboard editing tools: for example, to save your exclusion filters so they persist, or to build a variation scoped to a specific department or user group.

Reference: see the Consolidated Documentation Links at the end of this guide.

Discover Reporting

Location: Menu → Analytics → Discover

Discover is used for building ad hoc, in-depth reports beyond the dashboard's at-a-glance view.

Steps

1. Navigate to Analytics → Discover.
2. Confirm the data view is set to prompt (not logs).
3. Build your report by adding fields via the + icon next to each field in the sidebar. Common fields to include: event/timestamp, username, model, token count, request text, and response text.
4. Apply filters to exclude noise: username is not admin@ai.expedient.cloud, and model is not text-embedding-3-small.
5. Set your time range using a preset or a custom absolute/relative range.

Field Statistics

Use Field Statistics to pivot on data without building a full table:

• Distinct user count: useful for measuring adoption, such as comparing how many users have access versus how many are actively using the platform.
• Distinct model count: shows which models are actually in use across the organization.

Saving, Exporting, Scheduling, and Sharing

• Expand any individual record to view the full request and response text for detailed review.
• Save reports to your personal folder for one-click reuse; filters and time range are retained.
• Export results as a CSV for further analysis, such as in Excel, or for keyword and phrase searches across historical data.
• Schedule a report to run automatically on a recurring basis, such as daily at midnight; completed runs appear under Alerts for retrieval.
• Share a generated link so another admin or user can copy and adapt the same report for their own use.

Reference: see the Consolidated Documentation Links at the end of this guide.

Alerts

Location: Menu → Observability (or Stack Management) → Alerts

Alerts surface both scheduled report completions and rule-triggered notifications (see Rules, below, for rule configuration).

• Scheduled Discover reports will post notifications here when a new run completes and is ready to download.
• Alerts generated by configured rules (threshold breaches, keyword matches, and similar conditions) also appear here in real time.
• Optional external notification connectors: alerts can be configured to push to outside systems such as email, Microsoft Teams, Slack, Jira, or a custom webhook. Setup requires a credential handoff and is typically configured with support from your delivery or operations team.

A pre-built support alert connector may exist by default, notifying the operational support team if your Elastic instance becomes unhealthy. This is unrelated to your usage governance and requires no end-user action.

Reference: see the Consolidated Documentation Links at the end of this guide.

Rules

Location: Menu → Observability (or Stack Management) → Alerts → Manage Rules

Rules define the conditions that generate alerts. Two rule types are consistently relevant to compliance and governance use cases.

Custom Threshold Rule

Used to trigger an alert when a numeric metric (such as total tokens or prompt count) crosses a defined line.

1. Go to Manage Rules → Create Rule → Custom Threshold.
2. Select the prompt data view.
3. Define the aggregation, such as total token count or document/prompt count, and optionally group by a field such as username to monitor specific users or groups.
4. Set the threshold condition, such as "is above 50,000,000 tokens."
5. Set the time window the rule evaluates, such as a trailing 30 days.
6. Set the check interval: how often the rule runs. Avoid intervals more frequent than hourly for heavier queries to prevent performance lag; daily is typical for consumption monitoring.
7. Optionally attach a notification action/connector.
8. Save the rule.

Example use case: alert your organization when total monthly token consumption approaches a contracted credit threshold.

Elasticsearch Query Rule

Used for keyword or phrase-based monitoring of prompt or response content: a reactive governance control for sensitive or flagged terms.

1. Go to Manage Rules → Create Rule → Elasticsearch Query.
2. Define your query, such as request text contains "[keyword/phrase]."
3. Set the match threshold, such as "count is above or equal to 1."
4. Set the evaluation time window, such as a trailing 30 days, and the check interval, such as once daily.
5. Optionally attach a notification action/connector.
6. Save the rule.

Example use case: flag any prompt or response containing terms related to PII, client-confidential data, or other sensitive/restricted content, without blocking usage but ensuring visibility for governance purposes. This is a reactive complement to proactive front-end controls such as the PII filter covered in the Administrator Guide : the front-end filter can redact sensitive content before it reaches the model, while a rule like this notifies you after the fact that a redaction (or a related term) occurred, so you have both preventative and after-the-fact visibility.

General Notes

• Both rule types support in-line documentation links within the Kibana UI for further configuration guidance.
• Other Kibana stack rule types, such as error count, exist but are generally not relevant to AI usage governance; they pertain to the operational health of the Elastic instance itself.

No hard usage caps. No native hard usage caps or quotas exist at the gateway level; the platform will not automatically block a user upon reaching a token limit. Consumption beyond contracted amounts is billed as overage. Rules and alerts are therefore the primary mechanism for proactive usage governance, rather than hard enforcement.

Reference:

• Create a custom threshold rule (Observability 8.19)
• Elasticsearch query rule type
• Kibana 8.19 Guide: Alerting section

Summary Reference Table

Consolidated Documentation Links

• Kibana Guide (8.19), full index
• Kibana 8.19.0 Release Notes
• Custom threshold rule (Observability 8.19)
• Elasticsearch query rule type (Kibana 8.19)

The links above have been verified against Elastic's published documentation. For Dashboards, Discover, and Connector configuration specifically, this guide points to the main Kibana 8.19 documentation index rather than a specific sub-page, since Elastic reorganizes exact page URLs between releases. From the index, use the in-page navigation or search to reach the specific topic; Dashboards, Discover, and Alerting/Connectors each have their own section.