Docker COPY files using glob pattern?

There is a solution based on multistage-build feature:

FROM node:12.18.2-alpine3.11

WORKDIR /app
COPY ["package.json", "yarn.lock", "./"]
# Step 2: Copy whole app
COPY packages packages

# Step 3: Find and remove non-package.json files
RUN find packages \! -name "package.json" -mindepth 2 -maxdepth 2 -print | xargs rm -rf

# Step 4: Define second build stage
FROM node:12.18.2-alpine3.11

WORKDIR /app
# Step 5: Copy files from the first build stage.
COPY --from=0 /app .

RUN yarn install --frozen-lockfile

COPY . .

# To restore workspaces symlinks
RUN yarn install --frozen-lockfile

CMD yarn start

On Step 5 the layer cache will be reused even if any file in packages directory has changed.

Using Docker's new BuildKit executor it has become possible to use a bind mount into the Docker context, from which you can then copy any files as needed.

For example, the following snippet copies all package.json files from the Docker context into the image's /app/ directory (the workdir in the below example)

Unfortunately, changing any file in the mount still results in a layer cache miss. This can be worked around using the multi-stage approach as presented by @mbelsky, but this time the explicit deletion is no longer need.

# syntax = docker/dockerfile:1.2
FROM ... AS packages

WORKDIR /app/
RUN --mount=type=bind,target=/docker-context \
    cd /docker-context/; \
    find . -name "package.json" -mindepth 0 -maxdepth 4 -exec cp --parents "{}" /app/ \;

FROM ...

WORKDIR /app/
COPY --from=packages /app/ .

The mindepth/maxdepth arguments are specified to reduce the number of directories to search, this can be adjusted/removed as desirable for your use-case.

It may be necessary to enable the BuildKit executor using environment variable DOCKER_BUILDKIT=1, as the traditional executor silently ignores the bind mounts.

More information about BuildKit and bind bounds can be found here.

If you can't technically enumerate all the subdirectories at stake in the Dockerfile (namely, writing COPY packages/one/package.json packages/one/ for each one), but want to copy all the files in two steps and take advantage of Docker's caching feature, you can try the following workaround:

• Devise a wrapper script (say, in bash) that copies the required package.json files to a separate directory (say, .deps/) built with a similar hierarchy, then call docker build …
• Adapt the Dockerfile to copy (and rename) the separate directory beforehand, and then call yarn install --pure-lockfile…

All things put together, this could lead to the following files:

./build.bash:

#!/bin/bash

tag="image-name:latest"

rm -f -r .deps  # optional, to be sure that there is
# no extraneous "package.json" from a previous build

find . -type d \( -path \*/.deps \) -prune -o \
  -type f \( -name "package.json" \) \
  -exec bash -c 'dest=".deps/$1" && \
    mkdir -p -- "$(dirname "$dest")" && \
    cp -av -- "$1" "$dest"' bash '{}' \;
# instead of mkdir + cp, you may also want to use
# rsync if it is available in your environment...

sudo docker build -t "$tag" .

and

./Dockerfile:

FROM …

WORKDIR /usr/src/app

# COPY package.json .  # subsumed by the following command
COPY .deps .
# and not "COPY .deps .deps", to avoid doing an extra "mv"
COPY yarn.lock .
RUN yarn install --pure-lockfile

COPY . .
# Notice that "COPY . ." will also copy the ".deps" folder; this is
# maybe a minor issue, but it could be avoided by passing more explicit
# paths than just "." (or by adapting the Dockerfile and the script and
# putting them in the parent folder of the Yarn application itself...)

As mentioned in the official Dockerfile reference for COPY <src> <dest>

The COPY instruction copies new files or directories from <src> and adds them to the filesystem of the container at the path <dest>.

For your case

Each may contain wildcards and matching will be done using Go’s filepath.Match rules.

These are the rules. They contain this:

'*' matches any sequence of non-Separator characters

So try to use * instead of ** in your pattern.