-
Notifications
You must be signed in to change notification settings - Fork 204
Description
In 9.3.0-SNAPSHOT using the any output configuration with max_retries: -1 will cause the collector exit at startup.
The collector error logs that result (the final two in the logs below) are not obviously sourced from the collector and contain no "log":{"source" object.
{"log.level":"info","@timestamp":"2025-12-12T19:07:34.149Z","log.logger":"otel_manager","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/otel/manager.(*subprocessExecution).startCollector","file.name":"manager/execution_subprocess.go","file.line":134},"message":"supervised collector started with pid: 33738 and healthcheck port: 61826","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-12-12T19:07:34.778Z","message":"invalid configuration: exporters::elasticsearch/_agent-component/default: retry::max_retries should be non-negative","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-12-12T19:07:34.778Z","message":"exporters::elasticsearch/_agent-component/monitoring: retry::max_retries should be non-negative","ecs.version":"1.6.0"}#11448 improved this a bit and we now get the final logs from the collector identified as being source from the otel_manager but the original logs from the collector appear as if they are from Elastic Agent itself making it impossible to filter out collector logs.
{"log.level":"info","@timestamp":"2025-12-12T19:07:34.149Z","log.logger":"otel_manager","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/otel/manager.(*subprocessExecution).startCollector","file.name":"manager/execution_subprocess.go","file.line":134},"message":"supervised collector started with pid: 33738 and healthcheck port: 61826","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-12-12T19:07:34.778Z","message":"invalid configuration: exporters::elasticsearch/_agent-component/default: retry::max_retries should be non-negative","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-12-12T19:07:34.778Z","message":"exporters::elasticsearch/_agent-component/monitoring: retry::max_retries should be non-negative","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2025-12-12T19:07:34.785Z","log.logger":"otel_manager","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/otel/manager.(*OTelManager).Run","file.name":"manager/manager.go","file.line":295},"message":"collector exited with error (will try to recover in 159ns): exporters::elasticsearch/_agent-component/monitoring: retry::max_retries should be non-negative","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-12-12T19:07:34.785Z","log.logger":"otel_manager","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/otel/manager.(*OTelManager).Run","file.name":"manager/manager.go","file.line":239},"message":"collector recovery restarting, total retries: 2","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}All logs from the collector should include a "log":{"source":"elastic-otel-collector"} property. This potentially will be addressed by the binary split when the collector binary is separated from that of Elastic Agent.