Skip to content

fix: keep the developer's dependencies out of the build context - #5

Closed
jeroenrinzema wants to merge 1 commit into
brainpodnl:mainfrom
jeroenrinzema:fix/image-host-dependency-context
Closed

fix: keep the developer's dependencies out of the build context#5
jeroenrinzema wants to merge 1 commit into
brainpodnl:mainfrom
jeroenrinzema:fix/image-host-dependency-context

Conversation

@jeroenrinzema

Copy link
Copy Markdown
Collaborator

A Railpack build copies the developer's node_modules into the image on top of the one it just installed. On a laptop that means darwin/arm64 binaries — Tailwind's oxide, lightningcss — inside a linux/amd64 image, and next build fails on them with nothing in the output pointing at the cause.

Cause

Railpack's build step is:

"inputs": [{"step": "install"}, {"include": ["."], "local": true}]

The local context is layered after the install step, so the working copy wins over what the container built. I planted a marker and a fake native.darwin-arm64.node in a host node_modules; both landed in the image, over the container's own install.

Why nobody has a .dockerignore

A .dockerignore does prevent this and is honoured today. The trap is that node_modules is in every project's .gitignore, which looks like it should cover this — and does nothing here.

Measured with --no-cache. Without it BuildKit reuses the unfiltered copy and the difference disappears, which is how I first misread this:

in the project host node_modules copied into the image
neither yes
.dockerignore no
.gitignore yes
both no

Change

The generated plan carries the exclusions itself, so this holds with or without a .dockerignore, which is still honoured on top for anything project-specific.

node_modules and .venv are the two dependency trees Railpack's own install steps create in /app. Both are rebuilt from the lockfile inside the container, and a host copy of either is never valid for the target platform, so refusing them cannot cost a legitimate build anything. Exclusions the plan already carried are preserved.

Verification

Real plan from railpack prepare, host node_modules populated, no .dockerignore, --no-cache, linux/arm64:

  • app/node_modules/HOST_MARKER.txt — absent
  • app/node_modules/left-pad/native.darwin-arm64.node — absent
  • app/node_modules/left-pad/package.jsonpresent (the container's install survives)
  • app/package.jsonpresent (sources still copied)

Plus two unit tests over the plan transform: exclusions are added only to local inputs, and existing ones are kept.

Not fixed here

The context upload. The frontend transfers everything and filters afterwards — a cold build sends 83.91MB either way, whether the exclusions come from .dockerignore or from the plan, so a project with a large node_modules still pays to upload it. That one belongs upstream in Railpack.

Independent of #4 — different layer, no overlap; either can merge first.

Railpack's build step layers the build context over its own install step, so
whatever is in the working copy wins. A node_modules from the developer's
machine therefore lands on top of the one the container just installed, which
puts darwin/arm64 binaries -- Tailwind's oxide, lightningcss -- into a
linux/amd64 image. The build that follows fails on them, and nothing in the
output says why.

A .dockerignore prevents it and is honoured today, but almost nothing has one,
and the reason is a trap: node_modules is in every project's .gitignore, which
looks like it should cover this and does not. Verified with --no-cache, since
BuildKit caches the unfiltered copy and hides the difference:

    ignore file       host node_modules copied into the image
    none              yes
    .dockerignore     no
    .gitignore        yes
    both              no

So the plan carries the exclusions itself. node_modules and .venv are both
rebuilt from the lockfile inside the container, and a host copy of either is
never valid for the target platform, so refusing them cannot cost a build
anything. Any exclusions the plan already had are kept.

This does not shrink the context upload. The frontend transfers the whole
context and filters afterwards -- 83.91 MB either way in a cold build, whether
the exclusions come from .dockerignore or from the plan -- so that one belongs
upstream.
@jeroenrinzema

Copy link
Copy Markdown
Collaborator Author

Closing in favour of having the skill write a project-appropriate .dockerignore. Hardcoding node_modules and .venv in the CLI is a guess that will drift as Railpack's providers change, and it only addresses dependency trees — the same copy also puts .env files, keys, and anything else in the working tree into the shipped image, which a per-project ignore file covers and a fixed list never will.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant