Reduce allocations by using pooled memory and recycling memory streams

[stebet]

The RabbitMQ client currently allocates a lot of unnecessary memory and has a lot of GC overhead.

I'm currently working on a PR to reduce the allocations being made, and will probably introduce some *Async overloads as well as they help reduce lock contention.

Here is the current progress I have made with a test app that opens two connections. One connection bulk publishes 50000 messages in 100 message batches containing just an 4-byte integer as payload. The other connection receives those messages so it's mostly just measureing the frame serialize/deserialize overhead.

Before:

After:

I'll follow up this issue with discussions and link the PR once it's ready for review.

[stebet]

Some explanations:
The before uses the current NuGet release of the RabbitMQ.Client library.
The after uses code based on the latest master branch which is the 6.0 release.

[michaelklishin]

So a 25% reduction with this specific run. Looks promising!

[lechu445]

ralated to #452

[stebet]

I have got System.IO.Pipelines working on the socket connection as well, will push the PR soon for review and testing. Improvements are quite impressive, will add details and screenshots later :)

[stebet]

So here's where I'm at. Similar scenario as above, one sender, one receiver. Bulk Send 50000 messages in 500x100 message batches, but now with a 512 byte, 4kb and 16kb payloads.

Before

512 byte payload

4kb payload

16kb payload

After (using pooled arrays, recyclable memorystreams and System.IO.Pipelines)

512 byte payload

4kb payload

16kb payload

[stebet]

More progress :)

512 byte payloads

4kb payloads

16kb payloads

To summarize:
512 byte payloads are down from 556mb to 238mb (57% reduction)
4kb payloads are down from 1.96gb to 411mb (79% reduction)
16kb payloads are down from 7.14gb to 1.00 gb (86% reduction)

What's left
To finish up the PR I need to get all the tests to run which will require a little refactoring around catching the exceptions/errors that happen now that the Pipelines are taking care of reading and writing the sockets, as they are little harder to reach and parse. Once that's ready, I'll submit the PR for further work and discussions on what APIs (if any) might need to change.

[michaelklishin]

@stebet impressive 💪

[lukebakken]

This will be addressed by either #706 or #707