Is there analogue to multi-container K8s jobs in Prefect

[anton-daneyko-ultramarin]

We currently use K8s to run a job that with two containers: one container with a third party service and the other one with our code that uses that third party service.

I wonder if this sort of setup would be possible and how exactly? We already use Prefect for some "crony" stuff, but this one seems different. My flow would need the access to the third party service, which I theoretically could start manually (using K8s API), but this does not merit the switch to Prefect, since right now the lifetime management of the third party service is taken care of by K8s job.

[desertaxle]

Hey @anton-daneyko-ultramarin! I haven't tried this myself, but you should be able to achieve this by modifying the base job template of a Kubernetes work pool. You can edit any aspect of the job manifest used to start the Kubernetes job that executes your flow, including adding additional containers to the job. If you think that will work for your use case, I can give an example base job template that uses multiple containers.

If that doesn't seem like it would work, you could also look into adding an init container to your Prefect job that starts the third-party container before starting your flow container.

[anton-daneyko-ultramarin]

@desertaxle Thank you for your swift reply! Sounds like both things would be worth trying. I have a few questions though:

1. Does the first option imply that we would need a dedicated K8s worker, since AFAIR it's a one to one relationship between the work pool and worker?
2. I cannot understand how the second option is different from the first one. On which level of the configuration hierarchy could the init conatiner be added?

[anton-daneyko-ultramarin]

I would, of course, appreciate an example very much!