How to use the Olakai Monitoring Integration agent skill

What the skill makes the agent do

An agent skill is a folder of instructions that your AI agent reads when a task matches it. This one is written as an integration checklist. The agent works through it inside your codebase, so it edits your existing code rather than running as a separate tool.

The skill covers three jobs: adding a tracking event after each model call, configuring the SDK with your Olakai API key, and defining custom KPIs (business metrics such as items processed or success rates) so you can see how the AI performs over time.

What you need before installing

The skill only makes sense if you already have a working LLM application and an Olakai account. Check these requirements first.

The skill's SKILL.md lists the following requirements. Node.js and Python have minimum versions, which are listed in the source requirements.

• An existing, working AI agent or application that uses OpenAI, Anthropic or another LLM
• Node.js (for TypeScript) or Python (for Python), at the minimum versions listed in the requirements
• The Olakai CLI, installed and authenticated
• An Olakai API key for each agent
• The @olakai/sdk package (npm) or olakai-sdk package (pip)
• An existing LLM provider SDK and API key, such as OpenAI

How to install the skill

Option 1: download the skill ZIP from the skill's page in this directory and unzip it into your user skills folder, ~/.claude/skills/, so it is available across projects.

Option 2: unzip it into a single project's .claude/skills/ folder so it is available only in that project.

Option 3: in the Claude apps, open Settings, go to Capabilities, and upload the skill under Skills.

How to trigger the skill with a natural request

Once installed, describe the task in plain language. Name the thing you want, for example adding Olakai monitoring to an existing OpenAI call, and point the agent at the code that makes the LLM call. The agent should then follow the skill's steps in your project.

Be specific about your situation. Say which LLM provider you use, whether the code is a single call or a multi-step agent, and which business outcomes you care about. That information shapes the tracking events and KPIs the agent proposes.

• Add Olakai monitoring to the OpenAI call in src/support/reply.ts
• Wrap our Anthropic client with Olakai tracking and define KPIs for ticket resolution
• Help me validate the Olakai integration end-to-end for this agent

What the integration looks like in code

The core change is small. You initialise the Olakai config once at startup, keep your existing LLM call, and add a tracking call after it. The source's TypeScript example adds the SDK, reads the Olakai key from an environment variable, and sends the prompt, response, token count, user email and a task label.

The skill notes that tracking is fire-and-forget for the basic pattern. For multi-call tasks it recommends aggregating calls into a single event, and for streamed responses it recommends tracking after the stream completes.

• Pattern A: one LLM client used throughout the app, one tracking call after each completion
• Pattern B: several LLM calls per request, aggregated into one event
• Pattern C: streaming responses, tracked after the full response is assembled
• Pattern D: a third-party or local model, tracked with manual event calls

How custom KPIs and customData work

Custom KPIs are where most of the value is, according to the skill. Plan to configure at least two to four KPIs that answer the question of how you know the agent is performing well. KPIs are unique per agent, so a new agent needs its own KPI definitions even if another agent already has similar ones.

Only fields registered as CustomDataConfigs are processed. Unregistered customData fields are stored but cannot be used in KPIs. Numeric configs need numbers, not strings. Formula names are case-insensitive. Do not send fields Olakai already tracks automatically, such as timestamps, token counts, model and provider.

• Register each customData field as a CustomDataConfig before using it in a KPI
• Send numbers as numbers for NUMBER configs
• Send data you will use for KPIs or filtering only

Safety notes before you accept changes

The skill runs scripts and uses the network. The example tracking calls send prompts, responses and user emails to a third-party service, Olakai. Review every tracking call before accepting it, and make sure the data you send is allowed under your privacy and data policies.

Do not let the agent paste real API keys into code. Read keys from environment variables, as the examples do, and keep them out of version control. Remove any customData fields that contain personal data you do not need.

• Review each tracking call and the fields it sends
• Use environment variables for OLAKAI_API_KEY and provider keys
• Create a separate API key per agent with olakai agents create

Validate the integration and tips

The skill includes validation as part of the work, so ask the agent to confirm the integration end-to-end. Run your app with a test request and check that the event appears in Olakai before you rely on any KPI.

Keep a short list of the KPIs you planned and confirm each one has a registered field behind it. For a team, agree on the chatId and taskExecutionId conventions early so events from related calls group correctly.

• Start with a non-production agent and a test API key
• Use debug mode in development to see what the SDK is doing
• Pass a consistent chatId for multi-turn chat so conversation-level KPIs work