Add a single-model ollama image.

Yes, it's yet another huge image (26GB) :)

This image is useful to benchmark checkpoint/restore performance.
To fairly measure it, we need an image that has only one model
preloaded into it, as this mirrors the situation where someone might
want to use checkpoint/restore for an inference model where there is
only one model to serve.

This image uses mixtral as a medium-large model. The larger the model,
the more important the gains in checkpoint/restore performance, so
measuring with a large model provides a more representative and
significant result.

PiperOrigin-RevId: 613335037
This commit is contained in:
Etienne Perot
2024-03-06 14:10:26 -08:00
committed by gVisor bot
parent a76911efa9
commit 35cb2282f0
+15
View File
@@ -0,0 +1,15 @@
# https://hub.docker.com/r/ollama/ollama
FROM ollama/ollama:0.1.26
ENV PATH=$PATH:/usr/local/nvidia/bin:/bin/nvidia/bin
ENV OLLAMA_ORIGINS=*
ENV OLLAMA_HOST=0.0.0.0:11434
# Pre-install a single model.
RUN bash -c ' \
( ollama serve ) & serverpid="$!"; \
sleep 5; \
ollama pull mixtral:instruct && \
kill "$serverpid" && \
wait "$serverpid" \
'