WARNING: THIS SITE IS A MIRROR OF GITHUB.COM / IT CANNOT LOGIN OR REGISTER ACCOUNTS / THE CONTENTS ARE PROVIDED AS-IS / THIS SITE ASSUMES NO RESPONSIBILITY FOR ANY DISPLAYED CONTENT OR LINKS / IF YOU FOUND SOMETHING MAY NOT GOOD FOR EVERYONE, CONTACT ADMIN AT ilovescratch@foxmail.com
Skip to content

[Improvement] supports metrics that can reflect original traffic in extractor #419

@caiq1nyu

Description

@caiq1nyu

Motivations

Currently, the extractor's rps and bps metrics (such as: ExtractorPushed*) describe the metrics that have been processed and filtered by the program and are ready to be pushed to the queue. In business use, it is often necessary to pay attention to scenarios such as remote bandwidth flow, database pressure and so on, and the existing metrics are not enough to help analyze and troubleshoot problems.

Solution

Implements the metrics in task_metrics.rs:

  • ExtractorRpsMax,
  • ExtractorRpsMin,
  • ExtractorRpsAvg,
  • ExtractorBpsMax,
  • ExtractorBpsMin,
  • ExtractorBpsAvg,

Priority support for the following engines:

  • Postgres (struct, snapshot, cdc)
  • MySQL (struct, snapshot, cdc)
  • Redis (psync)
  • Mongo (snapshot, cdc)

Metadata

Metadata

Assignees

Labels

EnhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions