disable buildx on demand

[y-nk]

Description

is there a way to not use buildx and only have commands which are docker build ?

[crazy-max]

Do you mean building with legacy builder without BuildKit? Fyi this action only uses Buildx that only supports BuildKit.

[y-nk]

@crazy-max yes, i mean without buildkit. because the action is named "docker build and push" so i would expect that it's possible to build and push with the simplest docker commands (no buildx, no buildkit)

to be clear it's an information question, not a feature request. if i can, i would love to, if it's not possible, then i already found a workaround so it's not so much of a big deal.

[crazy-max]

yes, i mean without buildkit

This is not possible with this action that relies only on Buildx requiring BuildKit. Do you mind sharing why you don't want to use BuildKit? Do you have some limitations?

[y-nk]

@crazy-max not some limitations, but i was trying to keep my CI simple as my setup involves a lot of steps and caching.

i've a multi-stage rust image. the 1st stage is made to build dependencies and run unit tests, then the 2nd stage is used as production build. my ci does first a docker build --target 1st-stage (i simplified the command), then a docker run to run the 1st stage and the unit tests. after that, the next step is to run integration tests, for which the 2nd stage of the image is built within docker compose and i have a separate (nodejs) container as the test runner.

the image has a very big amount of dependencies, which builds in ~20mn in a regular runner so leveraging caching is key to avoid building 3 times the same part (1st deps, 2nd unit tests, 3rd intg tests) - that's exactly why i built this docker-based strategy rather than building locally in the runner.

i first wanted to rely on the internal registry directly, which worked great locally (since buildx is disabled in my default orbstack setup). in the ci tho, since this action provides a builder, we must add --cache-from= --cache-to= in the build command. we must also buildx bake the docker compose manifest along side an additional docker-compose.json file to wire the cache to docker compose prior docker compose up. on top, i couldn't make good use of type=gha because the build command should work locally too and also due to the limitation of sharing cache between workflow runs ; so i ended up using type=local and manage it myself.

all that just so that i can have a proper cache strategy when buildx is enabled, whereas it works out of the box when it's disabled. so i thought that while it may be suitable for many people in most of cases, it may also be useful sometimes to be able to disable it.

extract of dockerfile

### builder
FROM rust:1.85.0 AS builder

WORKDIR /usr/src/router

# install OS deps
RUN apt-get update
RUN apt-get install -y cmake
RUN apt-get install -y python3
RUN apt-get install -y protobuf-compiler
RUN apt-get install -y libprotobuf-dev

# build the deps (this step is used for docker caching, see readme for more explanations)
COPY Cargo.toml Cargo.lock rust-toolchain.toml ./
RUN rustup update
RUN mkdir src
RUN echo "fn main() {}" > src/main.rs
RUN cargo build --release

# force opt-out cache for lines below
ADD "https://www.random.org/cgi-bin/randbyte?nbytes=10&format=h" skipcache

# build the project
COPY . .
RUN cargo build --release

### project runner
FROM debian:12-slim

RUN apt-get update
RUN apt-get install --yes ca-certificates

WORKDIR /app
COPY --from=builder /usr/src/router/target/release/graphql-gateway-router ./

extract of task runner file

[env]
COMPOSE_DOCKER_CLI_BUILD = 1
DOCKER_BUILDKIT = 1

[tasks.build]
run = """
mkdir -p /tmp/.buildx
docker buildx build . \
  --cache-from "type=local,src=/tmp/.buildx" \
  --cache-to "type=local,dest=/tmp/.buildx,mode=max" \
  --tag "router-build:latest" \
  --target builder \
  --load
"""

[tasks.test]
run = [ 'mise run test-unit', 'mise run test-spec' ]

[tasks.test-unit]
hide = true
run = """
docker run router-build:latest cargo test --release;
"""

[tasks.test-spec]
hide = true
run = """
cd tests;
pnpm install --ignore-workspace;
pnpm test;
"""

[_]
docker-cache = [""]

extract of the tests/docker-compose.yml + docker-compose.json

services:
  router:
    build:
      context: ..
    command: ./graphql-gateway-router --config /router-config --supergraph /schema-config
    ports:
      - 8080:8080
    configs:
      - schema-config
      - router-config

configs:
  router-config:
    file: ./config/router-config.yaml

  schema-config:
    file: ./schema/supergraph.graphql

{
  "target": {
    "mock-api": {
      "cache-from": ["type=local,src=/tmp/.buildx"],
      "cache-to": ["type=local,dest=/tmp/.buildx,mode=max"],
      "output": ["type=docker"]
    },
    "router": {
      "cache-from": ["type=local,src=/tmp/.buildx"],
      "cache-to": ["type=local,dest=/tmp/.buildx,mode=max"],
      "output": ["type=docker"]
    }
  }
}

extract of tests/package.json (the intg test runner)

{
  "name": "graphql-gateway-tests",
  "scripts": {
    "pretest": "docker buildx bake --file docker-compose.yml --file docker-compose.json --allow=fs.read=.. --allow=fs.read=/tmp/.buildx --allow=fs.write=/tmp/.buildx && docker compose up --detach --wait; sleep 3;",
    "test": "vitest run"
  },
}

ci steps (highly edited)

name: Test

on:
  pull_request:

jobs:
  pr-check:
    runs-on: ubuntu-latest
    timeout-minutes: 120

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 1

      - uses: ./.github/actions/setup

      - uses: ./.github/actions/docker-cache

      - uses: docker/setup-buildx-action@v3

      - run: mise run lint

      - run: mise run build

      - run: mise run test

      - if: ${{ failure() }}
        working-directory: ${{ steps.setup.outputs.path }}
        run: docker compose logs
        continue-on-error: true

internal docker cache action

name: Docker
description: Build and push docker images

runs:
  using: composite
  steps:
    - id: cache-key
      run: |
        keys=$(mise config get _.docker-cache || echo '[]')
        echo "service docker cache keys: ${keys}"

        prefixed=$(echo "${keys}" | jq -r -c --arg path "." 'map("\($path)/" + .)')
        echo "result=${prefixed}" >> $GITHUB_OUTPUT
      shell: bash

    - id: hashed-key
      if: steps.cache-key.outputs.result != '[]'
      # we can't do better since hashFiles does not support passing an array so, we'll accept that 4 globs are a limitation. it should be enough to target: dockerfile, pnpm lock, cargo lock, and an extra glob.
      run: |
        cacheKey="${{ hashFiles(fromJson(steps.cache-key.outputs.result)[0]) }}-${{ hashFiles(fromJson(steps.cache-key.outputs.result)[1]) }}-${{ hashFiles(fromJson(steps.cache-key.outputs.result)[2]) }}-${{ hashFiles(fromJson(steps.cache-key.outputs.result)[3]) }}"

        echo "result=${cacheKey}" >> $GITHUB_OUTPUT
      shell: bash

    - if: steps.hashed-key.outputs.result != ''
      uses: actions/cache@v4
      with:
        path: /tmp/.buildx
        key: ${{ steps.hashed-key.outputs.result }}

[crazy-max]

since this action provides a builder, we must add --cache-from= --cache-to= in the build command.

In such case you can just specify the docker driver so build result is saved in docker store without overhead:

      - uses: docker/setup-buildx-action@v3
        with:
          driver: docker

[y-nk]

@crazy-max oh you're right, i've never considered this approach. but as i remember, the cache would need to be loaded in order to be used, rather than using --cache-from and --cache-to with type=local would be slightly faster as it doesn't need to be loaded, right?