Running migrate without starting the web server

[awgymer]

I am wondering if it would be possible to add a command and/or documentation around the best way (if it makes sense) to run migrate (or other manage.py commands) without actually starting the web server.

The rationale behind this is that there are some cases where you want to populate the database or run some other management command without having the actual web server running (such as things you want to ensure happen before any celery tasks are sent).

The best I could come up with so far was to do something like:

## Bring up just Redis and Postgres
COMPOSE_PROFILES=cache,db docker compose up 

## In another shell
docker compose run --no-deps --rm python3 manage.py migrate

Do you have any thoughts on whether this is an OK thing to do? Is there a better way to run some pre-start commands (which may take arbitrarily long).

[nickjj]

docker compose run web python3 manage.py migrate with no additional modifications should work.

While it does use the web container, the command gets overridden to run a migration instead of gunicorn.

The DB and Redis are both dependencies of web so they will get started if not already (although technically in this case Redis isn't a dependency). If you wanted to optimize for not starting Redis and only the DB, you could run COMPOSE_PROFILES=postgres docker compose run web python3 manage.py migrate to have it all rolled up into 1 command with no exports needed.

[nickjj]

As for running pre-start commands. It really depends on what they are doing.

You could put them into the entrypoint script but they will run whenever your container starts. Usually putting arbitrarily long running commands there isn't ideal, but it depends on your use case.

Alternatively exec'ing into your already running container to run 1 off commands or using run like above are other ways to do it.

[awgymer]

I'm actually fitting an existing app and just using the scaffold of the compose file and the run script and I found that without the redis service up I got an error from Django about it, so that might just be an issue of our projects config.

Is compose run able to know not to do anything with the DB/redis images after the run container exits (I want to --rm it regardless of the restart policy when I'm not using it for the web server proper).

Sorry if these Qs are a bit too generic docker compose issues

[nickjj]

I ran it here locally with this project successfully. The app boots:

$ COMPOSE_PROFILES=postgres docker compose run web python3 manage.py migrate
[+] Running 1/1
 ✔ Container hellodjango-postgres-1  Started                                                                                                                                                                                                                                0.3s
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
  No migrations to apply.

When the command finishes, the Postgres container will still be running which you can verify with docker compose ps or docker container ps. You can run docker compose stop after the command to stop all dependent services, or just docker compose stop postgres in this case. I'd be careful with this pattern though because if you have your whole stack running with docker compose up before you ran the migration, stopping postgres will break your app until you re-up postgres.

[nickjj]

With that said, if you're on a host with a managed database / Redis running outside of Docker then setting COMPOSE_PROFILES=web,worker in your .env on that host is a common pattern. This prevents your DB and Redis starting automatically so you should be able to run docker compose run web python3 manage.py migrate and not have to worry about anything else since there's no compose profile to override or dependent containers to stop afterwards.

If the above doesn't work out of the box (I'm not 100% sure if run will use COMPOSE_PROFILES in the same way up does) then you can run COMPOSE_PROFILES= docker compose run web python3 manage.py migrate to explicitly set the compose profile to avoid starting your DB. This command would be good to run too if you have a manged DB but run Redis through Docker Compose.

[awgymer]

So after some testing it seems that if I bring up the Redis and Postgres containers with COMPOSE_PROFILES=db,cache docker compose up and then run docker compose run --no-deps --rm web python3 manage.py mycmd I get the behaviour I wanted (the web container automatically removed on command completion).

And I can use DC="run --no-deps --rm" ./run manage <command> to get this behaviour through the run script, even if it is a bit hacky.

Then after my manual pre-start setup jobs (which can be very long-running >1hr) I can just COMPOSE_PROFILES=web,workers docker compose up and everything is online.

[nickjj]

That works. It's not too hacky. The DC env var is there to let you override which command to run and in your case you want to run things without dependencies and clean up the container afterwards.