# OpenTelemetry Collector presets

> Send the logs of your Docker containers or Kubernetes pods to dlogify without changing your images or your code.

By the end of this page, a collector on your host reads the logs of the containers you choose and sends them to dlogify. You change no image and no code: you add one service and a label.

The preset runs the official OpenTelemetry Collector image (`otel/opentelemetry-collector-contrib:0.161.0`) with a configuration file you can read. It joins stack traces into one event, detects levels and exceptions, and parses JSON lines, following the same rules as `logify run`, with the limits listed in [What the collector parses](#what-the-collector-parses).

## Docker

You need Docker Engine on Linux and a project API key (see [Projects and API keys](https://docs.dlogify.com/dashboard/projects-and-keys.md)).

### 1. Start the collector

Run one collector per host. In an empty directory, download the two files and start it:

```bash
curl -fsSLO https://docs.dlogify.com/collector/compose.yaml
curl -fsSLO https://docs.dlogify.com/collector/logify-collector.yaml
LOGIFY_API_KEY=lgf_live_0123_example LOGIFY_ENVIRONMENT=production docker compose up -d
```

You can also copy the `logify-collector` service and its volume into your own Compose file, next to `logify-collector.yaml`.

The collector reads `/var/lib/docker/containers` read-only. It keeps its read positions, and the logs dlogify has not accepted yet, in a volume: after a restart it continues where it stopped, so lines your running containers wrote in the meantime are sent. A container created while the collector is stopped is read from the moment the collector starts again. It runs as root, because only root can list Docker's log directories, with a read-only file system, no Linux capabilities and no open ports.

### 2. Opt your containers in

A container is collected only when it has the label `logify.enabled=true` **and** uses the `json-file` logging driver with the options below. The `labels` option copies the labels into every log line, which is how the collector knows which container a line belongs to.

```yaml
x-logify-logging: &logify-logging
  driver: json-file
  options:
    labels: logify.enabled,logify.service,com.docker.compose.service
    tag: "{{.Name}}"
    max-size: 10m
    max-file: "3"

services:
  api:
    image: my-org/api:1.4.2
    labels:
      logify.enabled: "true"
      logify.service: api
    logging: *logify-logging
```

Recreate the containers after changing labels or logging options (`docker compose up -d`). Logging options apply to new containers only.

To avoid repeating the logging options, set them once for the whole host in `/etc/docker/daemon.json` and restart Docker; then the label alone opts a container in:

```json
{
  "log-driver": "json-file",
  "log-opts": {
    "labels": "logify.enabled,logify.service,com.docker.compose.service",
    "tag": "{{.Name}}",
    "max-size": "10m",
    "max-file": "3"
  }
}
```

For `docker run`, pass `--label logify.enabled=true --log-opt labels=logify.enabled,logify.service,com.docker.compose.service --log-opt tag={{.Name}}`.

### Service and environment

| Attribute | Value |
|---|---|
| Service | the label `logify.service`, else the Compose service name, else the container name |
| Environment | `LOGIFY_ENVIRONMENT` of the collector (default `default`) |
| Container | `container.name` and `container.id` |

### Settings

| Variable | Default | Meaning |
|---|---|---|
| `LOGIFY_API_KEY` | required | Project API key |
| `LOGIFY_ENDPOINT` | `https://api.dlogify.com` | Where logs are sent |
| `LOGIFY_ENVIRONMENT` | `default` | Environment of every event this collector sends |

Docker Desktop on macOS and Windows has not been verified yet. If the collector sends nothing there, use the [Node.js](https://docs.dlogify.com/send-logs/node.md) or [Python](https://docs.dlogify.com/send-logs/python.md) SDK, or wrap your start command with [logify run](https://docs.dlogify.com/send-logs/node.md#any-process-logify-run).

## Kubernetes (preview)

The preset installs the official `open-telemetry/opentelemetry-collector` Helm chart (version 0.175.0) as a DaemonSet. It has been validated against the chart, not yet run on a production cluster.

```bash
curl -fsSLO https://docs.dlogify.com/collector/values.yaml
kubectl create namespace logify
kubectl -n logify create secret generic logify --from-literal=api-key=lgf_live_0123_example
helm install logify-collector opentelemetry-collector \
  --repo https://open-telemetry.github.io/opentelemetry-helm-charts \
  --version 0.175.0 -n logify -f values.yaml
```

Opt pods in with the annotation `dlogify.com/enabled: "true"` in the pod template. The service name follows the OpenTelemetry conventions: the annotation `resource.opentelemetry.io/service.name`, then the labels `app.kubernetes.io/instance` and `app.kubernetes.io/name`, then the workload name. Edit `LOGIFY_ENVIRONMENT` and `LOGIFY_ENDPOINT` under `extraEnvs` in `values.yaml`.

## What the collector parses

The collector applies the same parsing rules as `logify run`, except the ones that need to look back at earlier lines of the same event:

- Indented lines, `Caused by:` and `... N more` join the line before them, so Node.js, Java and similar stack traces arrive as one event with their exception type, message and stack trace.
- JSON lines are parsed: message, level, time, error and the other keys as attributes.
- Levels are detected from words such as `ERROR`, `[warn]` or `WARNING:` at the start of a line (after up to two timestamps), or an upper-case level word in the first 64 characters; a stack trace with no level is an error.
- Python tracebacks are split at the exception line that ends them, Go panics after the `panic:` line, and the excerpt Node.js prints before an uncaught exception arrives as separate events. For exact parsing of these, wrap the process with [logify run](https://docs.dlogify.com/send-logs/node.md#any-process-logify-run) instead.
- An event ends when a line that starts a new event arrives, or at the latest 500 ms after its first line, at 500 lines or at 40 KB. A stack trace that takes longer than 500 ms to be written can be split into several events.

## Traefik access logs

With Traefik's access log in JSON (`--accesslog=true --accesslog.format=json`), the collector turns each access line into a readable event:

- The message is `GET example.com/path -> 404`: method, host, path and the status the client received.
- The level comes from the status: `ERROR` for 5xx, `INFO` for anything else. Refused requests (404, 429, a client closing the connection) are counted in your INFO summaries; only server errors open error groups. To get error groups for 4xx too, set those events to `WARN` in your own collector pipeline after the dlogify one.
- The request is described with the standard attributes `http.request.method`, `http.response.status_code`, `url.path`, `server.address` and `user_agent.original`, plus `logify.http.router` with Traefik's router name. Traefik leaves request headers out of the access log by default; add `--accesslog.fields.headers.names.User-Agent=keep` so the user agent, which helps tell scanners from browsers, is sent.
- Every access line is an event and counts toward your plan's [rate limits and quotas](https://docs.dlogify.com/send-logs/limits.md#rate-limits-and-quotas), including successful requests. To send only refused and failed requests, add `--accesslog.filters.statuscodes=400-599`.

dlogify groups access logs by method, router and status, not by path: every path a bot tries is one group (`GET (unmatched) -> 404` when no router matched), and the paths stay visible in each event. Access logs also feed the project's [activity](https://docs.dlogify.com/dashboard/activity.md) charts and top routes.

## Troubleshooting

- Nothing arrives: check the container has both the label and the logging options (`docker inspect -f '{{.HostConfig.LogConfig}}' <container>`), then read the collector's own log with `docker compose logs logify-collector`. A `401` there means the API key is wrong or revoked.
- The service name is the container name: add the label `logify.service`, or check that the `labels` logging option lists `logify.service`.

Source: https://docs.dlogify.com/send-logs/collector
