Use when a user wants to wire an OpenTelemetry collector into a Managed ClickStack service on ClickHouse Cloud, either by deploying a new local collector (Docker run or Docker Compose) or by configuring their own existing collector, then send rich synthetic telemetry and verify it is visible in Clic

Set up an OpenTelemetry collector for Managed ClickStack This skill wires an OpenTelemetry collector into a Managed ClickStack service running on ClickHouse Cloud, sends rich synthetic telemetry through it, and confirms the data is actually visible in ClickStack. It uses [ clickhousectl ](https://clickhouse.com/docs/interfaces/cli) for all cloud and SQL operations. Scope. This skill supports two paths, chosen in Step 0: 1. Deploy a new collector locally. You can do this two ways: individual docker commands, or a docker compose file (recommended, fewer commands and one file to start/stop). Make the user aware of both up front and let them pick in Step 0; do not assume plain docker . Either way runs the ClickStack distribution of the collector, preconfigured for Managed ClickStack. 2. Configure your own existing collector by adding the ClickHouse exporter configuration. We give you the exact config to drop in; you reload your collector. Use this if you already run a collector in a gateway role. A full Kubernetes deployment (Helm, secrets in K8s Secrets) is out of scope here; the config we generate in path 2 can be applied to a collector running anywhere. The end state is: A dedicated hyperdx ingest SQL user on the target service, with exactly the grants the collector needs (it creates the otel. schema on first write). A collector forwarding logs, traces, and metrics into the otel database on the service, either the new local ClickStack collector or your existing one. Rich synthetic telemetry across several services, severities, span statuses, and metric types, so ClickStack's Search, Service Map, and dashboards have something real to show. The service confirmed awake , and the user walked through the ClickStack onboarding in the Cloud console so they can actually see their data. Secrets (the OTLP auth token and the SQL password) are generated locally, written once to a 0600 env file, and passed to Docker via env file . They are never pasted into the chat, never passed with docker run e , and never echoed back after creation. Follow these steps in order. Each step depends on state established by the previous one. Step 0: Choose your path Ask the user two short questions before doing anything else, because they determine which later steps run. Question 1: Do you already have an OpenTelemetry collector running in a gateway role? No, set one up for me. the new collector path. Continue to Question 2. Yes, I have one. the existing collector path. Skip Question 2 (it does not apply), and in Step 6 you will configure their collector rather than deploy a new one. Question 2 (new collector path only): Run the collector with individual Docker commands, or a Docker Compose file? Docker Compose (recommended). Fewer commands, one file to start and stop, easiest to re run. Best if docker compose is available. Individual Docker commands. Use if Compose is not installed or you prefer explicit commands. Record the answers as COLLECTOR PATH ( new or existing ) and, for the new path, DEPLOY MODE ( compose or run ). Refer back to them in Step 6 and Step 7. Step 1: Batch the permissions up front Coding agents prompt for approval the first time they see each shell command. To avoid interrupting the user every few steps, ask them once, up front, to allowlist the command prefixes below (the "always allow for this project / session" option in their agent). There are no destructive operations and nothing targets anything outside this project or their ClickHouse Cloud service. Command prefix Used for Needed when openssl rand … generate the OTLP token and SQL password always clickhousectl cloud … auth, resolve the service, run SQL via the Query API always jq … parse JSON from clickhousectl always docker … / docker compose … run/inspect the collector and the telemetry generator new collector path, and the optional telemetry check curl … local health check against localhost:13133 (and installing clickhousectl if missing) new collector path Tell the user, in your own words: "If your agent supports it, choose 'always allow' for each of these the first time it asks. The whole run is read only against your machine except for the collector container, and write operations against ClickHouse are limited to creating the ingest user and the otel schema." If the user is on the existing collector path and does not want to run the optional telemetry check, you can drop docker and curl from the list. Two approvals are semantic, not prefix based, so allowlisting won't pre clear them. Warn the user to expect these and approve them explicitly when they appear: The clickhousectl install in Step 3 uses curl … sh , which many agent sandboxes flag as "downloading and running untrusted code" regardless of any curl allowlist rule. The CREATE USER / GRANT in Step 5 may be flagged as "modifying shared production infrastructure," again independent of the clickhousectl prefix rule. Neither is solved by the table above; they are one time, intentional, and safe to approve. Then continue. Step 2: Confirm the target service and lay down the secrets file The user's prompt contains a service identifier, either a service ID (UUID) or a service name. Treat that value as SERVICE REF . Create a working directory and a 0600 env file that will hold all configuration and secrets for this run. The key names match exactly what the collector image reads, so this same file is passed straight to docker run env file (or referenced by Compose) in Step 6. Write it under a tight umask so the secret is never briefly world readable: Two things about these values matter and are easy to get wrong: Key names are exact. The collector reads CLICKHOUSE USER , CLICKHOUSE PASSWORD , CLICKHOUSE ENDPOINT , and HYPERDX OTEL EXPORTER CLICKHOUSE DATABASE . Store the SQL password under CLICKHOUSE PASSWORD (not a custom name); if it is missing, the collector starts with an empty password and dies with code: 516, Authentication failed . The password charset is constrained from three directions at once. ClickHouse Cloud rejects passwords without at least one uppercase character and one special character, so a plain hex string fails at CREATE USER . At the same time, the collector's migration tool embeds the password in a connection URL, so @ , : , / , ? , , and % corrupt it (symptom: code: 516 at startup even though the password is "correct"). The recipe above is random hex (lowercase + digits) plus the suffix Aa1 , which adds the required uppercase, a digit, and a URL unreserved special character ( ). The OTLP token has no such rules (it is just a bearer token), so plain hex is fine for it. The env file uses bare KEY=VALUE lines with no quotes : Docker's env file does not do shell parsing, so any quotes you add become part of the value. On the existing collector path the OTLP AUTH TOKEN is not used by your collector (auth on your receiver is your own setup); it is generated only so the same file works if you later switch to the local collector. The CLICKHOUSE values are still used: they go into the exporter config you add to your collector in Step 6. Every later step runs in a fresh shell, so WORKDIR , ENV FILE , and any exported credentials do not persist, and WORKDIR / ENV FILE are not stored inside the env file, so sourcing it can't recover them. Begin each subsequent step's shell with this standard preamble , which re derives the paths from the deterministic default, loads the saved credentials (Step 3), and loads the config: If you chose a non default WORKDIR , set it explicitly at the top of every step (the ${WORKDIR: …} default only covers the standard location). Later steps refer to this as "the standard preamble". Confirm with the user that SERVICE REF is correct. Tell them the working directory and that collector.env (mode 0600 ) now holds the OTLP token and the SQL password. Do not print either secret. If they want to see a value, point them at the file ( grep OTLP AUTH TOKEN "$ENV FILE" ). If the user supplied their own token or password, write those into the file instead of the generated ones, but keep the same 0600 discipline and make sure any custom password still meets the charset rules above. Step 3: Authenticate clickhousectl (separate terminal by default) Check clickhousectl is on PATH . Run this presence check on its own , not chained to the installer: the curl … sh form drags a harmless check into a compound command that sandboxes deny wholesale as an untrusted code download. Only if that prints nothing, install it (the user may need to approve this explicitly, see Step 1): Check authentication: This skill needs API key authentication : OAuth is read only and cannot create users or run write queries. If the API key row is not Active , the user must authenticate. Do not ask the user to paste their API key and secret into the chat. Anything pasted into the conversation lives in the transcript and has to be rotated afterward. Instead, ask them to authenticate in a separate terminal , then tell you when they are done: I need a ClickHouse Cloud Admin API key to create the ingest user and verify the data. Please don't paste it here. Instead: 1. In the [Cloud console](https://console.clickhouse.cloud), open Organization → API keys → New API key , and give it the Admin role. (Developer scoped keys can't provision the per service Query API endpoint that cloud service query uses.) 2. In a separate terminal , run: 3. Tell me when that's done and I'll re check the auth status. Poll until the API key row reports Active , then confirm with a real privileged call rather than trusting the status table alone. Use a ref agnostic call here: SERVICE REF may be a name, and cloud service get only accepts a UUID, so confirming with get would fail on a name for reasons unrelated to auth. cloud service list needs no ref and proves the API key works: If the list returns your services, you are authenticated; continue. The actual name or UUID resolution of SERVICE REF happens in Step 4. Expect to need the env var credentials (common, not an edge case). Many clickhousectl builds save the credentials file but a freshly spawned shell (such as the one your tool calls run in) doesn't read it, so auth status shows Active yet the very next clickhousectl call reports No credentials found . Rather than treat this as a rare fallback, write a small sourceable creds file once, then load it in every later shell. This keeps each subsequent shell to a single . line instead of two jq re derivations, and keeps the secret out of the chat: From now on, open every shell that calls clickhousectl with both loads , because env vars do not persist across shells: Re run the service list check above with the creds loaded; it should now succeed. Do not continue until a real call works. (If clickhousectl auth status already shows API key … Active and calls succeed without creds.env , you can skip this; but most agent shells need it.) Step 4: Resolve the service and capture the HTTPS endpoint Run the standard preamble (Step 2) so the paths, credentials, and config are all loaded in this shell, then resolve the service. If SERVICE REF is a UUID, use it directly; otherwise look it up by name: Extract the values you need, coercing the port to an integer. The port serializes as a float ( 8443.0 ); if :8443.0 leaks into the endpoint the collector's ClickHouse exporter cannot dial it: STATE must be running . If it is stopped or starting , ask the user to start the service (or wait), and do

clickstack-otel-collector