When telemetry goes missing, the OpenTelemetry Collector is usually the first place people look and often where they get stuck. The Collector sits between your applications and your backend, and a failure in any of its parts (receiver, pipeline, processor, exporter or queue) looks the same from the outside: no data.

If your Collector is being killed with exit code 137 or dropping data under load, see OpenTelemetry Collector OOMKilled: memory_limiter Explained.

This guide covers the 15 problems you are most likely to hit, and a method for working out which one you have. I did not write the error messages from memory. I reproduced each failure in a local Docker lab running Collector contrib v0.161.0 and Elasticsearch 9.5.2, and the messages below are copied from those runs. Where something is based on documentation rather than a lab run, I say so.

Short version

• Find the first hop where data stops. Do not change ten settings at once.
• If you leave out endpoint, the OTLP receiver listens on 127.0.0.1 only. Other containers and pods get connection refused.
• A receiver that is defined but not used in a pipeline is not started; in my v0.161.0 test, the Collector did not warn about the unused receiver.
• Port 4317 is OTLP over gRPC and 4318 is OTLP over HTTP. They are not interchangeable.
• Processors can drop data without logging anything. The Collector’s own metrics show where it disappeared.
• Data in Elasticsearch is not the same as data in the Kibana APM views.

The method: find the first hop where data stops

Every piece of telemetry travels the same path. Each arrow below is a place where it can stop.

Application (SDK)
    |  OTLP over gRPC (4317) or HTTP (4318)
    v
Receiver
    |
    v
Processors  (memory_limiter, filter, transform, batch ...)
    |
    v
Exporter + sending queue
    |  network, TLS, auth
    v
Backend (Elasticsearch, Tempo, a vendor ...)
    |
    v
UI (Kibana, Grafana ...)

Work through it in order and stop at the first “no”:

1. Is the application producing telemetry at all?
2. Can the application reach the Collector’s address and port?
3. Is the receiver listening, on the right interface and protocol?
4. Is the receiver part of a pipeline for the right signal?
5. Does data come out of the processors?
6. Does the exporter reach the backend and get accepted?
7. Is the data stored where your UI is looking?

The official Collector troubleshooting guide recommends the same thing for multi-hop pipelines: check each hop separately.

The lab setup

Everything ran in Docker on one user-defined network, so containers could reach each other by name. To send test data I used telemetrygen, the load generator from the Collector contrib repository:

docker network create otellab

docker run -d --name otel-collector --network otellab \
  -v "$PWD":/cfg \
  otel/opentelemetry-collector-contrib:0.161.0 --config=/cfg/config.yaml

docker run --rm --network otellab \
  ghcr.io/open-telemetry/opentelemetry-collector-contrib/telemetrygen:v0.161.0 \
  traces --otlp-endpoint otel-collector:4317 --otlp-insecure --traces 5

One practical note: the official contrib image has no shell, so you cannot docker exec into it and run curl. Run network tests from another container on the same network instead.

Before you start: turn on the Collector’s own metrics

The most useful troubleshooting tool is the Collector’s internal telemetry. It exposes Prometheus metrics on port 8888, but by default only on 127.0.0.1, so nothing outside the container can scrape it. To expose it:

service:
  telemetry:
    metrics:
      readers:
        - pull:
            exporter:
              prometheus:
                host: 0.0.0.0
                port: 8888

These are the metrics I used throughout this guide. All names below were confirmed on v0.161.0, which exposes them without a _total suffix:

• Ingress: otelcol_receiver_accepted_spans, otelcol_receiver_refused_spans, otelcol_receiver_failed_spans
• Processing: otelcol_processor_incoming_items, otelcol_processor_outgoing_items
• Egress: otelcol_exporter_sent_spans, otelcol_exporter_send_failed_spans
• Queue: otelcol_exporter_queue_size, otelcol_exporter_queue_capacity, otelcol_exporter_enqueue_failed_spans
• Process: otelcol_process_memory_rss, otelcol_process_cpu_seconds

Logs and metrics have _metric_points and _log_records versions of the span counters. Together they turn “is the Collector up?” into a more useful question: how many items came in, and how many left?

1. “connection refused”

What you see (client side):

dial tcp 172.21.0.2:4317: connect: connection refused

Why it happens: the client reached the host, but nothing was listening on that port. Common causes:

• The Collector is not running, or exited at startup because of a config error (see problem 5).
• Wrong port, or the port is not published from the container.
• The receiver listens on localhost only (problem 2) or is not in a pipeline (problem 3).
• The application sends to localhost while the Collector runs in a different container.

That last one is very common. Inside a container, localhost means that container itself. Pointing telemetrygen at localhost:4317 from a separate container gave:

dial tcp 127.0.0.1:4317: connect: connection refused

How to fix it: use the Collector’s service or container name (otel-collector:4317), and check that something is listening:

# on the host running the Collector
lsof -nP -iTCP:4317 -sTCP:LISTEN

# from the application's network (container or pod)
nc -vz otel-collector 4317

2. The receiver only listens on localhost

What you see: the Collector is running and the pipeline looks correct, but other containers or pods still get connection refused.

Why it happens: this config looks harmless:

receivers:
  otlp:
    protocols:
      grpc:
      http:

With no endpoint, current Collector versions bind to localhost. The startup log on v0.161.0 showed:

"endpoint": "127.0.0.1:4317"
"endpoint": "127.0.0.1:4318"

Anything outside the Collector’s own container is refused. Older tutorials written before this default changed often leave the endpoint out.

How to fix it: set the endpoint explicitly whenever clients connect from another container, pod or host:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

Binding to 0.0.0.0 exposes the port on every interface, so restrict access with network rules where that matters.

3. The receiver is defined but not in a pipeline

What you see: the Collector starts cleanly and logs Everything is ready. Begin running and processing data., yet clients get connection refused.

Why it happens: defining a component under receivers: does not start it. Per the configuration docs, a component is only enabled when a pipeline in the service section uses it. In this test config, otlp is defined but only a metrics pipeline exists, and it uses a different receiver:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
  hostmetrics:
    scrapers:
      load:
exporters:
  debug:
service:
  pipelines:
    metrics:
      receivers: [hostmetrics]   # otlp is never used
      exporters: [debug]

On v0.161.0 this produced no warning about the unused receiver, and port 4317 was never opened.

How to fix it: reference the receiver in a pipeline for each signal you expect. A receiver used only in a traces pipeline will not accept logs or metrics.

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug]
    logs:
      receivers: [otlp]
      exporters: [debug]

4. OTLP gRPC vs HTTP: port 4317 vs 4318

The OTLP specification sets the default ports: 4317 for gRPC and 4318 for HTTP. Send the wrong protocol to a port and the network works fine, but the protocols do not. The errors are distinctive, so they are worth recognising.

gRPC client sent to 4318 (the HTTP port):

error reading server preface: http2: frame too large, note that the frame
header looked like an HTTP/1.1 header

HTTP client sent to 4317 (the gRPC port):

Post "http://otel-collector:4317/v1/traces": net/http: HTTP/1.x transport
connection broken: malformed HTTP response "\x00\x00\x06\x04\x00..."

How to fix it: line up three things: the SDK exporter’s protocol, the receiver protocol, and the port. With the standard SDK environment variables:

# gRPC
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317

# HTTP (the SDK appends /v1/traces, /v1/metrics, /v1/logs)
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318

5. The Collector will not start: configuration errors

What you see: the container exits or restarts, and nothing listens on any port.

How to find it: validate the config before you debug the network. The binary name depends on your distribution (otelcol, otelcol-contrib and so on). With Docker you don’t need to know it:

docker run --rm -v "$PWD":/cfg \
  otel/opentelemetry-collector-contrib:0.161.0 validate --config=/cfg/config.yaml

Three real mistakes and what validate reported for each:

# typo in a pipeline reference (debg instead of debug)
Error: service::pipelines::traces: references exporter "debg" which is not configured

# typo in a config key (htp instead of http)
'protocols' has invalid keys: htp

# memory_limiter declared with no settings
Error: processors::memory_limiter: 'check_interval' must be greater than zero

The last one catches a lot of people. Many examples show memory_limiter: on its own line, but it has required settings and the Collector refuses to start without them (see problem 11).

One more thing to look for in startup logs: recent releases renamed some components. On v0.161.0 the old names still work but log a warning:

"otlp" alias is deprecated; use "otlp_grpc" instead
"otlphttp" alias is deprecated; use "otlp_http" instead

These renames apply to the exporters. The otlp receiver keeps its name. On older Collector versions, otlp_grpc and otlp_http do not exist yet, so match the names to the version you run.

6. “context deadline exceeded”

What you see: I pointed an exporter at a non-routable address with a 2 second timeout. The Collector logged:

info  Exporting failed. Will retry the request after interval.
  {"error": "rpc error: code = DeadlineExceeded desc = context deadline exceeded
   while waiting for connections to become ready", "interval": "7.02426119s"}

Note the level: info, not error. While the exporter is still retrying, the failure is logged at info. If you only search the logs for errors, you will miss it. Search for Exporting failed instead.

Why it happens: an operation did not finish within its timeout. Compare it with problem 1. connection refused means the host answered and said no. A deadline error usually means no answer arrived in time: the packets were dropped by a firewall, the route is wrong, or the backend is too slow.

How to fix it: first work out which hop timed out. If the error is in your application’s logs, the problem is between the app and the Collector. If it is in the Collector’s exporter logs, it is between the Collector and the backend. Then test that hop from the same network:

docker run --rm --network otellab curlimages/curl -v --max-time 5 https://your-backend:443

Don’t just raise the timeout. If the backend is slow because it is overloaded, a longer timeout only moves the failure to the queue (problem 13).

7. TLS errors

What you see: I set up an OTLP receiver with a certificate signed by a private CA. A client that did not trust that CA got:

tls: failed to verify certificate: x509: certificate signed by unknown authority

A client sending plaintext gRPC to the same TLS port got a different error:

error reading server preface: EOF

In both cases the Collector logged nothing at the default log level. TLS failures show up on the client side, so look there first.

Why it happens: there are usually two separate TLS connections, and each one has its own certificates:

Application  --TLS #1-->  Collector receiver
Collector exporter  --TLS #2-->  Backend

How to fix it: on the receiver side, serve a certificate whose name matches the hostname clients use (see the TLS setup docs):

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
        tls:
          cert_file: /certs/server.crt
          key_file: /certs/server.key

On the client side, give it the CA. For an SDK that is OTEL_EXPORTER_OTLP_CERTIFICATE=/certs/ca.crt. For a Collector exporter talking to a backend with a private CA:

exporters:
  otlp_grpc:
    endpoint: backend.internal:4317
    tls:
      ca_file: /certs/ca.crt

The CA file must be readable by the user the Collector runs as. insecure_skip_verify: true is fine for a quick lab test, but it is not a production fix.

8. Authentication failures (401 and 403)

What you see: I ran the Elasticsearch exporter with a wrong password:

error  Exporting failed. Dropping data.
  {"error": "Permanent error: flush failed (401): ...", "dropped_items": 10}

Why it matters: note the word Permanent. Unlike a timeout, an authentication failure is not retried. The data is dropped straight away, and fixing the credentials later will not bring it back. A 401 means the credentials are wrong. A 403 means they are valid but lack the privileges to write to that index or data stream.

How to fix it: test reachability and credentials separately, outside the Collector:

# can I reach it, and do these credentials work?
curl -u elastic:"$PASSWORD" https://your-elasticsearch:9200/
curl -H "Authorization: ApiKey $ELASTIC_API_KEY" https://your-elasticsearch:9200/

Keep secrets out of the config file. The Collector expands environment variables:

exporters:
  elasticsearch:
    endpoint: ${env:ELASTIC_ENDPOINT}
    api_key: ${env:ELASTIC_API_KEY}

9. The exporter cannot reach the backend

What you see (a typo in the Elasticsearch hostname):

"error": "failed to execute the request: dial tcp: lookup es-typo on 127.0.0.11:53: no such host"

How to isolate it: this is where the debug exporter earns its place. Add it to the same pipeline as your real exporter:

exporters:
  debug:
    verbosity: detailed
  otlp_grpc/backend:
    endpoint: backend:4317

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug, otlp_grpc/backend]

Every exporter in a pipeline gets a copy of the data, so the two can be compared directly. In my run the metrics made the split obvious:

# exporter "debug"
otelcol_exporter_sent_spans         10

# exporter "otlp_grpc/broken"
otelcol_exporter_send_failed_spans  10

If the debug exporter prints your spans, everything up to the exporter works and the problem is downstream: DNS, firewall, proxy, TLS or auth. If the debug exporter prints nothing, the problem is upstream, in the receiver or the processors. Remove the debug exporter when you are done, because detailed output is very noisy.

10. Data disappears inside the pipeline

What you see: the receiver accepts data, the exporter sends nothing, and there are no errors anywhere.

Why it happens: processors can filter, sample or drop data, and that is not an error, so it is not logged. To demonstrate, I added a filter processor with a condition that happened to match my test service:

processors:
  filter/drop-telemetrygen:
    error_mode: ignore
    traces:
      span:
        - resource.attributes["service.name"] == "telemetrygen"

The receiver accepted 20 spans, the debug exporter printed nothing, and the logs had no warning. The processor metrics showed exactly where the spans went:

# receiver "otlp"
otelcol_receiver_accepted_spans          20

# processor "memory_limiter"
otelcol_processor_incoming_items         20
otelcol_processor_outgoing_items         20

# processor "filter/drop-telemetrygen"
otelcol_processor_incoming_items         20
otelcol_processor_outgoing_items          0
otelcol_processor_filter_spans_filtered  20

How to fix it: look for the processor where incoming is greater than outgoing. If you don’t have metrics, cut the pipeline down to receiver -> debug exporter and add processors back one at a time until the data disappears.

11. Processors in the wrong order

Processors run in the order you list them in the pipeline, and the order matters. The processor README recommends:

1. memory_limiter first, so it can refuse data before anything else buffers it.
2. Sampling and filtering, to drop unwanted data early.
3. Processors that need request context, before batching, because batching clears that context.
4. Transformation and enrichment, applied only to data that will be exported.
5. batch last. The README also says to prefer the exporter’s own batching where it is available.

The memory_limiter needs real settings. Per its README, check_interval defaults to 0 (1s is recommended), and you must set either limit_mib or limit_percentage. A configuration that validates:

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 20
  batch:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp_grpc]

12. Batching makes data look late, or lost

What you see: you send one test span, check the backend straight away, see nothing, and assume it was lost.

Why it happens: batching holds data back on purpose. The batch processor sends when it reaches send_batch_size (default 8192) or when timeout passes (default 200ms). Some exporters batch on their own as well. The Elasticsearch exporter‘s default queue flushes after 10s or at 1MB, whichever comes first. In my lab a single test burst took several seconds to show up in Elasticsearch.

How to fix it: wait at least one flush interval before deciding data is lost, and check otelcol_exporter_sent_spans instead of refreshing the UI. For a test environment you can lower the timeouts. In production, leave batching on, because it reduces requests and improves compression.

13. “sending queue is full”

What you see: to force this, I gave an exporter a tiny queue (queue_size: 5) and an unreachable backend. The Collector logged:

error  Exporting failed. Rejecting data.  {"error": "sending queue is full", "rejected_items": 100}

The rejection went all the way back to the client:

traces export: exporter export timeout: rpc error: code = Unavailable desc = sending queue is full

The metrics told the whole story:

otelcol_exporter_queue_size            5
otelcol_exporter_queue_capacity        5
otelcol_exporter_enqueue_failed_spans  2500
otelcol_receiver_refused_spans         2500

Why it happens: the Collector receives data faster than it can export it. The queue fills, new data is refused, and the receiver passes the refusal back to clients. A queue that is full and stays full means the backend or the network is the bottleneck, not the queue size.

How to fix it: work out why exports are slow or failing first (problems 6 to 9). Then size the queue for short outages, tune num_consumers, and scale Collectors horizontally if the steady-state load is too high. The options are documented in the exporterhelper README, which also covers a persistent queue backed by storage so queued data survives a restart. Remember that a bigger queue also uses more memory.

14. Memory pressure and restarts

What you see: Collector restarts, OOMKilled in Kubernetes, memory climbing steadily, or refused data under load. I did not push the lab to an out-of-memory state, so this section is based on the memory_limiter README.

How the memory limiter behaves:

• Above the soft limit (limit minus spike limit), it refuses new data. The receiver passes the refusal back so clients can retry.
• Above the hard limit, it also forces garbage collection.
• Once memory drops below the soft limit, it accepts data again.

How to fix it: the memory limiter protects the process. It does not add capacity. Watch otelcol_process_memory_rss together with the queue metrics. If memory is high because the queue is full, the real fix is downstream (problem 13). Otherwise give the Collector more memory or add replicas. When you use limit_percentage in a container, make sure the container has a memory limit for the percentage to apply to.

15. Data reached Elasticsearch, but you cannot find it

This one is two problems that often get confused: the data is not where you are looking, or it is there but the UI you want cannot use it.

Where the data actually went. According to its README, the Elasticsearch exporter has used the otel mapping mode by default since v0.122.0. That mode requires Elasticsearch 8.12 or later and works best on 8.16 or later. Traces go to the data stream traces-generic.otel-default unless you route them elsewhere. In my lab:

GET _cat/indices/traces-*?h=index,docs.count&expand_wildcards=all

.ds-traces-generic.otel-default-2026.10.04-000001 10

The documents keep OpenTelemetry’s structure (attributes.*, resource.attributes.*, trace_id, span_id), not the field names you might know from older Elastic APM agents. Collectors older than v0.122.0 defaulted to a different mapping mode, with a different index and different field names. So when you upgrade a Collector, dashboards and saved queries can quietly stop matching.

If Elasticsearch returns a 400 or a mapping error, check these four things together before assuming the telemetry is malformed: the exporter version, the Elasticsearch version, the mapping mode, and the target data stream.

Why the APM views can stay empty. Writing spans into Elasticsearch is not the same as filling the Kibana APM UIs, which expect extra derived fields and metrics. The exporter README recommends enriching data with the elasticapm processor so documents contain the fields the Kibana UI needs. That processor (and the matching connector) is not part of the upstream contrib build; I checked the v0.161.0 component list. It ships in Elastic’s own Collector distribution.

Quick checks in Kibana: make sure your data view matches the data stream (traces-* covers both naming styles), widen the time picker, and search by a known trace_id.

Kubernetes: extra places to check

Kubernetes adds its own networking layer, so a healthy Collector can still look broken. I did not run a cluster for this guide. The checks below come from the official Kubernetes documentation.

# is the Service there, and does it have ready backends?
kubectl get svc -n observability otel-collector
kubectl get endpointslices -n observability -l kubernetes.io/service-name=otel-collector

# is the Collector healthy?
kubectl logs -n observability deploy/otel-collector
kubectl describe pod -n observability -l app.kubernetes.io/name=otel-collector

# test from the application's own network namespace
kubectl debug -it <app-pod> --image=busybox:1.36 --target=<app-container> -- \
  nc -vz otel-collector.observability.svc.cluster.local 4317

The last command uses an ephemeral debug container. That matters because the official Collector image has no shell. Things to check:

• No endpoints: the Service selector does not match the Collector pod labels, or targetPort is wrong. See Debug Services.
• Name does not resolve: across namespaces, use the full name <service>.<namespace>.svc.cluster.local. See Debugging DNS Resolution.
• Times out but resolves: a NetworkPolicy may be blocking ingress to the Collector or egress from the app.
• Refused from other pods: the receiver is bound to localhost (problem 2).

A minimal debugging configuration

When a production config is too complex to reason about, swap in this one. If the debug exporter prints your spans, the path from the application to the Collector works. Then add your processors and real exporter back one at a time.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug]

A production starting point

This passes validate on v0.161.0. Treat it as a starting point and adapt it to your distribution, version, load and security requirements.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 20
  batch:

exporters:
  otlp_grpc:              # "otlp" on Collector versions before the rename
    endpoint: backend.example.com:4317

service:
  telemetry:
    metrics:
      readers:
        - pull:
            exporter:
              prometheus:
                host: 0.0.0.0
                port: 8888
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp_grpc]

Troubleshooting checklist

• Application: instrumentation active, correct endpoint, protocol matches the port, not using localhost across containers.
• Network: DNS resolves, port reachable from the app’s network, no firewall, proxy or NetworkPolicy in the way.
• Receiver: explicit 0.0.0.0 endpoint, right protocol enabled, used in a pipeline for the right signal.
• Config: validate passes, memory_limiter has settings, no deprecated names in the startup logs.
• Processors: recommended order, incoming vs outgoing items match what you expect.
• Exporter: backend reachable, TLS trusted, credentials valid with the right privileges.
• Capacity: queue not pinned at capacity, no enqueue failures, memory and CPU have headroom.
• Backend and UI: right data stream, data view and time range, and the enrichment your UI needs.

Frequently asked questions

Why is my OpenTelemetry Collector not receiving data?

The most common causes are: the receiver listens on localhost because no endpoint is set, the receiver is not used in any pipeline, the client sends gRPC to the HTTP port (or the reverse), or the client uses localhost from a different container. Check them in that order.

What is the difference between OTLP port 4317 and 4318?

4317 is the default port for OTLP over gRPC, and 4318 is the default for OTLP over HTTP. A gRPC client must use the gRPC port and an HTTP client the HTTP port. Each mismatch produces its own error, shown in problem 4.

How do I see if the Collector is dropping data?

Expose the Collector’s internal metrics on port 8888 and compare otelcol_receiver_accepted_spans with otelcol_exporter_sent_spans. To find the exact processor that drops data, compare otelcol_processor_incoming_items and otelcol_processor_outgoing_items for each processor.

Why does “context deadline exceeded” not appear as an error in Collector logs?

While the exporter is still retrying, the Collector logs the failure at info level with the message “Exporting failed. Will retry the request after interval.” Search the logs for “Exporting failed” rather than filtering on error level.

Official references

• OpenTelemetry: Collector troubleshooting
• OpenTelemetry: Collector internal telemetry
• OpenTelemetry: Collector configuration
• OpenTelemetry: OTLP specification
• Collector processors README (recommended order)
• Memory limiter processor README
• Exporter helper README (queue and retry)
• Elasticsearch exporter README
• Kubernetes: Debug running pods
• Kubernetes: Debug Services

You can also read this article on Medium.

Connect with me on LinkedIn.

Related articles:

• OpenTelemetry to Elastic End to End: What Actually Breaks
• OpenTelemetry MCP Tracing to Elasticsearch: What Actually Works