Modules: Concurrency issue with module context fix #4603
Conversation
| // We use interface as we do not know exactly how the modules will use their inner context. | ||
| type ModuleContext map[string]interface{} | ||
| type ModuleContext struct { | ||
| sync.RWMutex |
There was a problem hiding this comment.
Discussed in eng subcommittee. Would it be better to use a sync map?
There was a problem hiding this comment.
I think that the only two phases related to bidders that cause parallelism of hook group (of one bidder) updating the ModuleInvocationContext while another bidder's hooks are reading. I mention this because I believe that, if a RWMutex is used only those two hooks would need to hold the read lock while calling the hooks, so the efficiency would only be exchanged for safety where it matters (if we used sync.Map, then all hook stages would take the performance hit).
https://docs.prebid.org/prebid-server/developers/add-a-module.html
Add holdReadLock mechanism to prevent concurrent map access in moduleContexts during parallel bidder hook execution. This fixes panics caused by concurrent reads and writes to the moduleContexts map when multiple bidders execute hooks in parallel. This is an alternate implementation to PR prebid#4603 (prebid#4603), using a simpler approach with a boolean flag to conditionally hold the read lock for the entire duration of hook execution in bidder stages. Changes: - Add holdReadLock boolean field to executionContext - Add getModuleContextLocked() helper that reads without acquiring locks - Modify executeGroup() to conditionally hold RLock for entire execution - Enable holdReadLock for ExecuteBidderRequestStage and ExecuteRawBidderResponseStage The RWMutex is now held for the entire duration that hook goroutines are executing in bidder stages, preventing saveModuleContexts() from acquiring write lock while reads are in progress. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <[email protected]>
|
Please see #4631 as an alternative approach, which uses the read-lock when calling the parallel bidder hook stages. NOTE: I wrote it with Claude code and need to review myself - it may not be exactly what I wanted - I may want it to have better control over the lock than it currently does. I just walked through it - it seems fine - each of the hookExecutor methods seems to call saveModuleContexts AFTER calling the executeStage and therefore holding a single lock for the duration of the executeGroup is fine - I might have grabbed and held at each hook invocation, but conceptually you can't return until all hooks are called so it's perfectly fine to only have one read lock held, so 👍 |
3 participants
#4239