fix: keep the developer's dependencies out of the build context - #5
Closed
jeroenrinzema wants to merge 1 commit into
Closed
fix: keep the developer's dependencies out of the build context#5jeroenrinzema wants to merge 1 commit into
jeroenrinzema wants to merge 1 commit into
Conversation
Railpack's build step layers the build context over its own install step, so
whatever is in the working copy wins. A node_modules from the developer's
machine therefore lands on top of the one the container just installed, which
puts darwin/arm64 binaries -- Tailwind's oxide, lightningcss -- into a
linux/amd64 image. The build that follows fails on them, and nothing in the
output says why.
A .dockerignore prevents it and is honoured today, but almost nothing has one,
and the reason is a trap: node_modules is in every project's .gitignore, which
looks like it should cover this and does not. Verified with --no-cache, since
BuildKit caches the unfiltered copy and hides the difference:
ignore file host node_modules copied into the image
none yes
.dockerignore no
.gitignore yes
both no
So the plan carries the exclusions itself. node_modules and .venv are both
rebuilt from the lockfile inside the container, and a host copy of either is
never valid for the target platform, so refusing them cannot cost a build
anything. Any exclusions the plan already had are kept.
This does not shrink the context upload. The frontend transfers the whole
context and filters afterwards -- 83.91 MB either way in a cold build, whether
the exclusions come from .dockerignore or from the plan -- so that one belongs
upstream.
Collaborator
Author
|
Closing in favour of having the skill write a project-appropriate .dockerignore. Hardcoding |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A Railpack build copies the developer's
node_modulesinto the image on top of the one it just installed. On a laptop that means darwin/arm64 binaries — Tailwind's oxide, lightningcss — inside a linux/amd64 image, andnext buildfails on them with nothing in the output pointing at the cause.Cause
Railpack's build step is:
The local context is layered after the install step, so the working copy wins over what the container built. I planted a marker and a fake
native.darwin-arm64.nodein a hostnode_modules; both landed in the image, over the container's own install.Why nobody has a
.dockerignoreA
.dockerignoredoes prevent this and is honoured today. The trap is thatnode_modulesis in every project's.gitignore, which looks like it should cover this — and does nothing here.Measured with
--no-cache. Without it BuildKit reuses the unfiltered copy and the difference disappears, which is how I first misread this:node_modulescopied into the image.dockerignore.gitignoreChange
The generated plan carries the exclusions itself, so this holds with or without a
.dockerignore, which is still honoured on top for anything project-specific.node_modulesand.venvare the two dependency trees Railpack's own install steps create in/app. Both are rebuilt from the lockfile inside the container, and a host copy of either is never valid for the target platform, so refusing them cannot cost a legitimate build anything. Exclusions the plan already carried are preserved.Verification
Real plan from
railpack prepare, hostnode_modulespopulated, no.dockerignore,--no-cache,linux/arm64:app/node_modules/HOST_MARKER.txt— absentapp/node_modules/left-pad/native.darwin-arm64.node— absentapp/node_modules/left-pad/package.json— present (the container's install survives)app/package.json— present (sources still copied)Plus two unit tests over the plan transform: exclusions are added only to
localinputs, and existing ones are kept.Not fixed here
The context upload. The frontend transfers everything and filters afterwards — a cold build sends
83.91MBeither way, whether the exclusions come from.dockerignoreor from the plan, so a project with a largenode_modulesstill pays to upload it. That one belongs upstream in Railpack.Independent of #4 — different layer, no overlap; either can merge first.