Motivations
Currently, the extractor's rps and bps metrics (such as: ExtractorPushed*) describe the metrics that have been processed and filtered by the program and are ready to be pushed to the queue. In business use, it is often necessary to pay attention to scenarios such as remote bandwidth flow, database pressure and so on, and the existing metrics are not enough to help analyze and troubleshoot problems.
Solution
Implements the metrics in task_metrics.rs:
- ExtractorRpsMax,
- ExtractorRpsMin,
- ExtractorRpsAvg,
- ExtractorBpsMax,
- ExtractorBpsMin,
- ExtractorBpsAvg,
Priority support for the following engines:
- Postgres (struct, snapshot, cdc)
- MySQL (struct, snapshot, cdc)
- Redis (psync)
- Mongo (snapshot, cdc)