Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker multiplatform build support #1974

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

PathToLife
Copy link

Adds multiplatform docker support for arm64 and armv7 architectures. Power efficient.. hopefully🔋🌲

Github workflow will need a test. However working locally with docker buildx and test image has been published below.

pathtolife/cyberchef:v10.19.4

image

https://hub.docker.com/r/pathtolife/cyberchef/tags

Command to run and remove on stop (--rm)

docker run --rm -p 8080:80 pathtolife/cyberchef:v10.19.4

image

Thanks!

@CLAassistant
Copy link

CLAassistant commented Feb 13, 2025

CLA assistant check
All committers have signed the CLA.

@PathToLife PathToLife force-pushed the feature/docker-multiplatform-build branch from 050c1c3 to f8b613b Compare February 13, 2025 07:58
@a3957273
Copy link
Member

a3957273 commented Feb 15, 2025

This was the initial implementation of the build job, but ended up taking very long. Over an hour to build an ARM build on the AMD build workers. Is there any chance you could test on a local repository how long the GitHub runners take to build the single image before this PR, and how long it would take to build all three?

This does have the benefit of only building on releases, which I'm less worried about having long build times. However, would still enjoy specific timings before merging this in.

@PathToLife
Copy link
Author

Over an hour to build an ARM build on the AMD build workers. Is there any chance you could test on a local repository how long the GitHub runners take to build the single image before this PR, and how long it would take to build all three?

Will report back. I'll add the time consuming build step only runs on one platform (amd64)! (so we won't be building on arm).

The multiplatform build step will be taking the amd64 build files and putting them in crossplatform supported nginx containers.

image

@a3957273
Copy link
Member

Oh, that's actually a really cool solution. Never mind then, the previous implementation also built on every other platform, which was incredibly slow. This seems far better. Don't worry about timing it, I'll be happy to test it out when I get time.

@PathToLife
Copy link
Author

PathToLife commented Feb 25, 2025

I had triggered a release last week. However it seems the CI OS image ubuntu-latest has updated the version of chrome. Therefore the UI Tests step fails, reporting the chrome versions are incompatiable with the npm driver.

You might have similar issues on the next release.

https://github.com/PathToLife/CyberChef/actions/runs/13350228747/job/37285773548

value: {
       error: 'session not created',
       message: 'session not created: This version of ChromeDriver only supports Chrome version 130\n' +
         'Current browser version is 133.0.6943.53 with binary path /opt/google/chrome/chrome',
       stacktrace: ''
     }

image

@a3957273
Copy link
Member

a3957273 commented Mar 9, 2025

I was hoping this PR had fixed this issue, which made it into mainline around a month ago:

https://github.com/gchq/CyberChef/pull/1972/files

@PathToLife
Copy link
Author

I was hoping this PR had fixed this issue, which made it into mainline around a month ago:

https://github.com/gchq/CyberChef/pull/1972/files

Looks like releases.yml is missing that change. I'll apply it and give it a go.

- name: Install
run: |
npm ci
npm run setheapsize

@PathToLife PathToLife force-pushed the feature/docker-multiplatform-build branch from 22007ef to b85036b Compare March 10, 2025 04:28
@PathToLife
Copy link
Author

Is there a reason we are using buildah? I'd recommend using docker buildx in the CI step as it's more simple. If we want to stick with buildah, it might be better to manually generate tags per arch, and then merge them together to a docker pullable manifest.

Manifest comparison redhat-actions/buildah-build vs docker buildx

// ghcr.io/pathtolife/cyberchef:10.19.5-beta13 is built using Github CI buildah image
// note the arm64v8 tag
$> docker buildx imagetools inspect ghcr.io/pathtolife/cyberchef:10.19.5-beta13
  Name:      ghcr.io/pathtolife/cyberchef:10.19.5-beta13
  MediaType: application/vnd.oci.image.index.v1+json
  Digest:    sha256:23d696642893987c93c0a14559130674b47b5f708870580403f2911710c06c84
  
  Manifests:
    Name:      ghcr.io/pathtolife/cyberchef:10.19.5-beta13@sha256:88f1cf2883e07355d234693114eb7b06e2238d59282330ba850451b040d29c0c
    MediaType: application/vnd.oci.image.manifest.v1+json
    Platform:  linux/amd64
  
    Name:      ghcr.io/pathtolife/cyberchef:10.19.5-beta13@sha256:4fa3f62e9f8268b631ec7f458488958ffad712e0cef66c2f6e6f6a36795b015b
    MediaType: application/vnd.oci.image.manifest.v1+json
    Platform:  linux/arm64v8
    
// docker.io/pathtolife/cyberchef:v10.19.4 is built using buildx
$> docker buildx imagetools inspect pathtolife/cyberchef:v10.19.4
  Name:      docker.io/pathtolife/cyberchef:v10.19.4
  MediaType: application/vnd.oci.image.index.v1+json
  Digest:    sha256:0579538a0091c6501862c4ea071b311d09cdbb03f5fae431d7021178f6b2c804
  
  Manifests:
    Name:        docker.io/pathtolife/cyberchef:v10.19.4@sha256:dbf0a7ede085045bc2e89c8d38510734d475571d2697461d87eebc38bda5dd92
    MediaType:   application/vnd.oci.image.manifest.v1+json
    Platform:    linux/amd64
  
    Name:        docker.io/pathtolife/cyberchef:v10.19.4@sha256:039fe128f6661f0ee3a003c26b7c92f08771163cde21d3b3e6bfd80bdfcf4223
    MediaType:   application/vnd.oci.image.manifest.v1+json
    Platform:    linux/arm64
...(Truncated Output)

User docker pull experience

It's not ideal as standard pull on arm64 server fails

// Standard pull on arm64 server fails as it does not identify listed platform linux/arm64v8
$> docker pull ghcr.io/pathtolife/cyberchef:10.19.5-beta13
  10.19.5-beta13: Pulling from pathtolife/cyberchef
  no matching manifest for linux/arm64/v8 in the manifest list entries
  
// However a pull on same tag via sha256 works
$> ghcr.io/pathtolife/cyberchef:10.19.5-beta13@sha256:4fa3f62e9f8268b631ec7f458488958ffad712e0cef66c2f6e6f6a36795b015b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants