[PATCH 1/2] Dockerfile: Populate a pip cache
Given the number of jobs in CI we have which use python and pip install packages, we should do this once in the Dockerfile, in order to populate the cache. We let each job continue to create and use the virtual environments they need to facilitate making updates to these environments easier.
Signed-off-by: Tom Rini trini@konsulko.com --- tools/docker/Dockerfile | 13 +++++++++++++ 1 file changed, 13 insertions(+)
diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile index bd02531be249..27205002005c 100644 --- a/tools/docker/Dockerfile +++ b/tools/docker/Dockerfile @@ -265,6 +265,19 @@ RUN echo uboot ALL=NOPASSWD: ALL > /etc/sudoers.d/uboot RUN useradd -m -U uboot USER uboot:uboot
+# Populate the cache for pip to use +RUN wget -O /tmp/pytest-requirements.txt https://source.denx.de/u-boot/u-boot/-/raw/master/test/py/requirements.txt +RUN wget -O /tmp/sphinx-requirements.txt https://source.denx.de/u-boot/u-boot/-/raw/master/doc/sphinx/requirements.tx... +RUN virtualenv -p /usr/bin/python3 /tmp/venv && \ + . /tmp/venv/bin/activate && \ + pip install -r /tmp/pytest-requirements.txt \ + -r /tmp/sphinx-requirements.txt && \ + deactivate && \ + rm -rf /tmp/venv /tmp/pytest-requirements.txt /tmp/sphinx-requirements.txt +#RUN pip download -r /tmp/pytest-requirements.txt \ +# -r /tmp/sphinx-requirements.txt && \ +# rm -f /tmp/pytest-requirements.txt /tmp/sphinx-requirements.txt + # Create the buildman config file RUN /bin/echo -e "[toolchain]\nroot = /usr" > ~/.buildman RUN /bin/echo -e "kernelorg = /opt/gcc-12.2.0-nolibc/*" >> ~/.buildman
Signed-off-by: Tom Rini trini@konsulko.com --- .azure-pipelines.yml | 2 +- .gitlab-ci.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/.azure-pipelines.yml b/.azure-pipelines.yml index 5594a67d6b52..71b6ba4bcf01 100644 --- a/.azure-pipelines.yml +++ b/.azure-pipelines.yml @@ -2,7 +2,7 @@ variables: windows_vm: windows-2019 ubuntu_vm: ubuntu-22.04 macos_vm: macOS-12 - ci_runner_image: trini/u-boot-gitlab-ci-runner:jammy-20230308-21Mar2023 + ci_runner_image: trini/u-boot-gitlab-ci-runner:jammy-20230308-22Mar2023 # Add '-u 0' options for Azure pipelines, otherwise we get "permission # denied" error when it tries to "useradd -m -u 1001 vsts_azpcontainer", # since our $(ci_runner_image) user is not root. diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5431bf6011a6..dbf80ca676a8 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -10,7 +10,7 @@ default:
# Grab our configured image. The source for this is found # in the u-boot tree at tools/docker/Dockerfile -image: ${MIRROR_DOCKER}/trini/u-boot-gitlab-ci-runner:jammy-20230308-21Mar2023 +image: ${MIRROR_DOCKER}/trini/u-boot-gitlab-ci-runner:jammy-20230308-22Mar2023
# We run some tests in different order, to catch some failures quicker. stages:
On Thu, 23 Mar 2023 at 08:19, Tom Rini trini@konsulko.com wrote:
Signed-off-by: Tom Rini trini@konsulko.com
.azure-pipelines.yml | 2 +- .gitlab-ci.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
Reviewed-by: Simon Glass sjg@chromium.org
Hi Tom,
On Thu, 23 Mar 2023 at 08:19, Tom Rini trini@konsulko.com wrote:
Given the number of jobs in CI we have which use python and pip install packages, we should do this once in the Dockerfile, in order to populate the cache. We let each job continue to create and use the virtual environments they need to facilitate making updates to these environments easier.
Signed-off-by: Tom Rini trini@konsulko.com
tools/docker/Dockerfile | 13 +++++++++++++ 1 file changed, 13 insertions(+)
Reviewed-by: Simon Glass sjg@chromium.org
diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile index bd02531be249..27205002005c 100644 --- a/tools/docker/Dockerfile +++ b/tools/docker/Dockerfile @@ -265,6 +265,19 @@ RUN echo uboot ALL=NOPASSWD: ALL > /etc/sudoers.d/uboot RUN useradd -m -U uboot USER uboot:uboot
+# Populate the cache for pip to use +RUN wget -O /tmp/pytest-requirements.txt https://source.denx.de/u-boot/u-boot/-/raw/master/test/py/requirements.txt +RUN wget -O /tmp/sphinx-requirements.txt https://source.denx.de/u-boot/u-boot/-/raw/master/doc/sphinx/requirements.tx... +RUN virtualenv -p /usr/bin/python3 /tmp/venv && \
. /tmp/venv/bin/activate && \pip install -r /tmp/pytest-requirements.txt \-r /tmp/sphinx-requirements.txt && \deactivate && \rm -rf /tmp/venv /tmp/pytest-requirements.txt /tmp/sphinx-requirements.txt+#RUN pip download -r /tmp/pytest-requirements.txt \ +# -r /tmp/sphinx-requirements.txt && \ +# rm -f /tmp/pytest-requirements.txt /tmp/sphinx-requirements.txt
What are those lines for? Can you add a comment?
# Create the buildman config file RUN /bin/echo -e "[toolchain]\nroot = /usr" > ~/.buildman RUN /bin/echo -e "kernelorg = /opt/gcc-12.2.0-nolibc/*" >> ~/.buildman -- 2.34.1
Regards, Simon
On Fri, Mar 24, 2023 at 07:28:32AM +1300, Simon Glass wrote:
Hi Tom,
On Thu, 23 Mar 2023 at 08:19, Tom Rini trini@konsulko.com wrote:
Given the number of jobs in CI we have which use python and pip install packages, we should do this once in the Dockerfile, in order to populate the cache. We let each job continue to create and use the virtual environments they need to facilitate making updates to these environments easier.
Signed-off-by: Tom Rini trini@konsulko.com
tools/docker/Dockerfile | 13 +++++++++++++ 1 file changed, 13 insertions(+)
Reviewed-by: Simon Glass sjg@chromium.org
diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile index bd02531be249..27205002005c 100644 --- a/tools/docker/Dockerfile +++ b/tools/docker/Dockerfile @@ -265,6 +265,19 @@ RUN echo uboot ALL=NOPASSWD: ALL > /etc/sudoers.d/uboot RUN useradd -m -U uboot USER uboot:uboot
+# Populate the cache for pip to use +RUN wget -O /tmp/pytest-requirements.txt https://source.denx.de/u-boot/u-boot/-/raw/master/test/py/requirements.txt +RUN wget -O /tmp/sphinx-requirements.txt https://source.denx.de/u-boot/u-boot/-/raw/master/doc/sphinx/requirements.tx... +RUN virtualenv -p /usr/bin/python3 /tmp/venv && \
. /tmp/venv/bin/activate && \pip install -r /tmp/pytest-requirements.txt \-r /tmp/sphinx-requirements.txt && \deactivate && \rm -rf /tmp/venv /tmp/pytest-requirements.txt /tmp/sphinx-requirements.txt+#RUN pip download -r /tmp/pytest-requirements.txt \ +# -r /tmp/sphinx-requirements.txt && \ +# rm -f /tmp/pytest-requirements.txt /tmp/sphinx-requirements.txt
What are those lines for? Can you add a comment?
Oh heck, I sent the wrong two patches from the branch, that's why. Thanks.
participants (2)
-
Simon Glass -
Tom Rini