Required Hardware and Software<\/h3>\n\n- OS:<\/strong> Windows 10\/11<\/li>\n
- WSL2:<\/strong> Ubuntu 22.04 or later (e.g., Ubuntu 24.04)<\/li>\n
- Docker Desktop:<\/strong> Latest version (with WSL2 integration enabled)<\/li>\n
- NVIDIA GPU:<\/strong> For GPU-equipped PCs, install the NVIDIA Container Toolkit to utilize the GPU<\/li>\n
- Other:<\/strong> Internet connection, sufficient RAM (capacity depending on the LLM model size)<\/li>\n<\/ul>\n
Prerequisites<\/h3>\n\n- \n
Installing WSL2<\/strong>
Run the following command in PowerShell (Administrator mode) to install WSL2 and the Ubuntu distribution.<\/p>\n\n\nwsl --install -d Ubuntu-24.04<\/pre>\n<\/div>\n<\/div>\n<\/li>\n
- \n
Installing Docker Desktop<\/strong>
Download and install Docker Desktop on Windows from the official website<\/a>, and enable WSL2 integration.
\nIf you encounter the following error when running docker-compose ps<\/code> in your WSL2 environment, try restarting Docker Desktop. For some reason, this fixes it.
\nThe command 'docker-compose' could not be found in this WSL 2 distro. We recommend to activate the WSL integration in Docker Desktop settings. For details about using Docker Desktop with WSL 2, visit: https:\/\/docs.docker.com\/go\/wsl2\/
\n<\/code>Reference: How to resolve “The command ‘docker-compose’\uff5e” even when WSL integration is enabled in Docker Desktop<\/a> (Japanese)<\/p>\n<\/li>\n- \n
Installing the NVIDIA Container Toolkit (when using a GPU)<\/strong>
Perform the installation on Ubuntu on WSL2 using the following steps.<\/p>\n# Install required packages\nsudo apt-get update && sudo apt-get install -y curl gnupg ca-certificates\n\n# Add NVIDIA's GPG key and repository\ncurl -fsSL https:\/\/nvidia.github.io\/libnvidia-container\/gpgkey | sudo gpg --dearmor -o \/usr\/share\/keyrings\/nvidia-container-toolkit-keyring.gpg\ncurl -s -L https:\/\/nvidia.github.io\/libnvidia-container\/stable\/deb\/nvidia-container-toolkit.list \\\n | sed 's#deb https:\/\/#deb [signed-by=\/usr\/share\/keyrings\/nvidia-container-toolkit-keyring.gpg] https:\/\/#g' \\\n | sudo tee \/etc\/apt\/sources.list.d\/nvidia-container-toolkit.list\nsudo apt-get update\nsudo apt-get install -y nvidia-container-toolkit\n\n# Configure Docker to use the NVIDIA runtime\nsudo nvidia-ctk runtime configure --runtime=docker\nsudo systemctl restart docker\n<\/pre>\n<\/li>\n<\/ol>\nInstalling and Configuring Each Tool (Verification)<\/h2>\n
Here, we verify the operation of each tool. If you prefer, you can skip ahead to launching via docker-compose.<\/p>\n
Introducing ollama<\/h3>\n
ollama is a tool for running LLM inference locally.
Below is an example of starting an ollama container without GPU support.<\/p>\n
docker run -d \\\n -v ollama:\/root\/.ollama \\\n -p 11434:11434 \\\n --name ollama \\\n ollama\/ollama:0.6.2<\/pre>\nBelow is an example of starting a GPU-enabled ollama container.<\/p>\n
docker run -d --gpus=all<\/span> \\\n -v ollama:\/root\/.ollama \\\n -p 11434:11434 \\\n --name ollama \\\n ollama\/ollama:0.6.2<\/pre>\nTips:<\/em><\/p>\n\n- Mount a volume using the
-v<\/code> option to persist model data and other files.<\/li>\n- Use
--gpus=all<\/code> to configure GPU utilization.<\/li>\n- Initial model downloads and initial loading may take some time, so please allow yourself plenty of time.<\/li>\n
- Here, ollama version 0.6.2 is used.<\/li>\n<\/ul>\n
Using gemma3<\/h3>\n
gemma3 is one of the LLM models compatible with ollama.
You can start gemma3 inference by running the following command inside the ollama container.
\ngemma3 is available in model sizes with 1B \/ 4B \/ 12B \/ 27B parameters.
\ngemma3:1b is sufficient for operational verification, so we will use that. Those with abundant PC resources are welcome to try gemma3:4b, gemma3:12b, or gemma3:27b.<\/p>\n
docker exec -it ollama ollama run gemma3:1b<\/pre>\nTips:<\/em><\/p>\n\n- Since downloading the model may require several gigabytes of data, please perform this in a stable network environment.<\/li>\n
- It has been reported that the initial launch is “slow on first load,” so you may want to consider configuring caching or resident settings (e.g., adjusting
OLLAMA_KEEP_ALIVE<\/code>).<\/li>\n- On an environment with an NVIDIA GeForce RTX 3060 (12GB GPU memory) and 32GB system memory, gemma3:12b was the limit. Continuing to use 12b caused overheating and a Blue Screen of Death, so 4b seems like the safer choice for this specification.<\/li>\n<\/ul>\n
Startup Management with docker-compose<\/h2>\n
Here is how to use docker-compose to simultaneously start and manage multiple containers (ollama, open-webui, etc.).
Below is an example of a sample docker-compose.yml<\/code> file.<\/p>\nservices:\n ollama:\n image: ollama\/ollama\n container_name: ollama\n ports:\n - \"11434:11434\"\n volumes:\n - .\/ollama_data:\/root\/.ollama\n deploy:\n resources:\n reservations:\n devices:\n - driver: nvidia\n count: all\n capabilities: [gpu]\n\n open-webui:\n image: ghcr.io\/open-webui\/open-webui:main\n container_name: open-webui\n ports:\n - \"3000:8080\"\n volumes:\n - .\/open-webui_data:\/app\/backend\/data\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n depends_on:\n - ollama\n<\/pre>\nTips:<\/em><\/p>\n\n- If you encounter the error message
WARN[0000] \/path\/to\/docker-compose.yml: `version` is obsolete<\/code>, please delete the version line in your docker-compose file. The version line has been omitted in the example above. Apparently, version is no longer required in Docker Compose V2.
\nReference: [Docker Compose] How to resolve the warning `version` is obsolete<\/a> (Japanese)<\/li>\n- If you encounter the error message
Error saving credentials: error storing credentials - err: fork\/exec \/usr\/bin\/docker-credential-desktop.exe: exec format error, out: ``<\/code>, reinstalling Docker Desktop will fix it.
\nReference: Error saving credentials: error storing credentials \u2013 err: fork\/exec \/usr\/bin\/docker-credential-desktop.exe: exec format error, out: \u201c \u00b7 Issue #14039 \u00b7 docker\/for-win<\/a><\/li>\n- Configure
volumes<\/code> to persist data for each container in host folders.<\/li>\n- Use
depends_on<\/code> to explicitly state that open-webui depends on ollama.<\/li>\n- Adjust environment variables and additional settings as needed.<\/li>\n<\/ul>\n
How to Use<\/h2>\n
Enter localhost:3000 in your browser, register an account, and the chat screen will appear.
\n![\"Local](\"https:\/\/donguri3.net\/wp-content\/uploads\/2025\/03\/eef7bf3496a44e9271d38060f6495614-e1742388246567-1024x796.png\")
<\/a>It’s telling lies to me…
\n![\"Local](\"https:\/\/donguri3.net\/wp-content\/uploads\/2025\/03\/1b20081e9a83b57dd0f54ba96ead759c.png\")
<\/a><\/p>\nIt also looks like the GPU is being utilized.<\/p>\n
References<\/h2>\n\n- Running Gemma3 with Docker in a Local Environment (Ollama+OpenWebUI)<\/a> (Japanese)<\/li>\n
- Setting Up Ollama on Windows Using WSL2 and Docker<\/a> (Japanese)<\/li>\n
- Windows | Docker Docs<\/a><\/li>\n
- Installing the NVIDIA Container Toolkit \u2014 NVIDIA Container Toolkit<\/a><\/li>\n
- How to resolve “The command ‘docker-compose’\uff5e” even when WSL integration is enabled in Docker Desktop<\/a> (Japanese)<\/li>\n
- [Docker Compose] How to resolve the warning `version` is obsolete<\/a> (Japanese)<\/li>\n
- Error saving credentials: error storing credentials \u2013 err: fork\/exec \/usr\/bin\/docker-credential-desktop.exe: exec format error, out: \u201c \u00b7 Issue #14039 \u00b7 docker\/for-win<\/a><\/li>\n<\/ul>\n
Conclusion<\/h2>\n
In this article, we explained how to deploy ollama, open-webui, and gemma3 to a local environment using Docker Desktop (docker-compose) and the NVIDIA Container Toolkit under a WSL2 environment.
We covered everything from installing each tool to managing multiple containers using docker-compose, complete with concrete command and configuration examples.
This makes local LLM operation straightforward, enabling rapid development and secure data management.<\/p>\n
Feel free to customize it to match your own environment and build an efficient LLM setup.<\/p>\n","protected":false},"excerpt":{"rendered":"
In recent years, the demand for LLMs (Large Language Models) has rapidly grown, drawing significant attention […]<\/p>\n","protected":false},"author":4,"featured_media":970,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_locale":"en_US","_original_post":"https:\/\/donguri3.net\/?p=955","footnotes":""},"categories":[1169],"tags":[18,19,51,52,100,365],"class_list":["post-4994","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-local-llm","tag-docker","tag-docker-compose","tag-git","tag-https","tag-ubuntu-22-04","tag-wsl2","en-US"],"_links":{"self":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/4994","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/comments?post=4994"}],"version-history":[{"count":1,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/4994\/revisions"}],"predecessor-version":[{"id":4997,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/4994\/revisions\/4997"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/media\/970"}],"wp:attachment":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/media?parent=4994"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/categories?post=4994"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/tags?post=4994"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Prerequisites<\/h3>\n\n- \n
Installing WSL2<\/strong>
Run the following command in PowerShell (Administrator mode) to install WSL2 and the Ubuntu distribution.<\/p>\n\n\nwsl --install -d Ubuntu-24.04<\/pre>\n<\/div>\n<\/div>\n<\/li>\n
- \n
Installing Docker Desktop<\/strong>
Download and install Docker Desktop on Windows from the official website<\/a>, and enable WSL2 integration.
\nIf you encounter the following error when running docker-compose ps<\/code> in your WSL2 environment, try restarting Docker Desktop. For some reason, this fixes it.
\nThe command 'docker-compose' could not be found in this WSL 2 distro. We recommend to activate the WSL integration in Docker Desktop settings. For details about using Docker Desktop with WSL 2, visit: https:\/\/docs.docker.com\/go\/wsl2\/
\n<\/code>Reference: How to resolve “The command ‘docker-compose’\uff5e” even when WSL integration is enabled in Docker Desktop<\/a> (Japanese)<\/p>\n<\/li>\n- \n
Installing the NVIDIA Container Toolkit (when using a GPU)<\/strong>
Perform the installation on Ubuntu on WSL2 using the following steps.<\/p>\n# Install required packages\nsudo apt-get update && sudo apt-get install -y curl gnupg ca-certificates\n\n# Add NVIDIA's GPG key and repository\ncurl -fsSL https:\/\/nvidia.github.io\/libnvidia-container\/gpgkey | sudo gpg --dearmor -o \/usr\/share\/keyrings\/nvidia-container-toolkit-keyring.gpg\ncurl -s -L https:\/\/nvidia.github.io\/libnvidia-container\/stable\/deb\/nvidia-container-toolkit.list \\\n | sed 's#deb https:\/\/#deb [signed-by=\/usr\/share\/keyrings\/nvidia-container-toolkit-keyring.gpg] https:\/\/#g' \\\n | sudo tee \/etc\/apt\/sources.list.d\/nvidia-container-toolkit.list\nsudo apt-get update\nsudo apt-get install -y nvidia-container-toolkit\n\n# Configure Docker to use the NVIDIA runtime\nsudo nvidia-ctk runtime configure --runtime=docker\nsudo systemctl restart docker\n<\/pre>\n<\/li>\n<\/ol>\nInstalling and Configuring Each Tool (Verification)<\/h2>\n
Here, we verify the operation of each tool. If you prefer, you can skip ahead to launching via docker-compose.<\/p>\n
Introducing ollama<\/h3>\n
ollama is a tool for running LLM inference locally.
Below is an example of starting an ollama container without GPU support.<\/p>\n
docker run -d \\\n -v ollama:\/root\/.ollama \\\n -p 11434:11434 \\\n --name ollama \\\n ollama\/ollama:0.6.2<\/pre>\nBelow is an example of starting a GPU-enabled ollama container.<\/p>\n
docker run -d --gpus=all<\/span> \\\n -v ollama:\/root\/.ollama \\\n -p 11434:11434 \\\n --name ollama \\\n ollama\/ollama:0.6.2<\/pre>\nTips:<\/em><\/p>\n\n- Mount a volume using the
-v<\/code> option to persist model data and other files.<\/li>\n- Use
--gpus=all<\/code> to configure GPU utilization.<\/li>\n- Initial model downloads and initial loading may take some time, so please allow yourself plenty of time.<\/li>\n
- Here, ollama version 0.6.2 is used.<\/li>\n<\/ul>\n
Using gemma3<\/h3>\n
gemma3 is one of the LLM models compatible with ollama.
You can start gemma3 inference by running the following command inside the ollama container.
\ngemma3 is available in model sizes with 1B \/ 4B \/ 12B \/ 27B parameters.
\ngemma3:1b is sufficient for operational verification, so we will use that. Those with abundant PC resources are welcome to try gemma3:4b, gemma3:12b, or gemma3:27b.<\/p>\n
docker exec -it ollama ollama run gemma3:1b<\/pre>\nTips:<\/em><\/p>\n\n- Since downloading the model may require several gigabytes of data, please perform this in a stable network environment.<\/li>\n
- It has been reported that the initial launch is “slow on first load,” so you may want to consider configuring caching or resident settings (e.g., adjusting
OLLAMA_KEEP_ALIVE<\/code>).<\/li>\n- On an environment with an NVIDIA GeForce RTX 3060 (12GB GPU memory) and 32GB system memory, gemma3:12b was the limit. Continuing to use 12b caused overheating and a Blue Screen of Death, so 4b seems like the safer choice for this specification.<\/li>\n<\/ul>\n
Startup Management with docker-compose<\/h2>\n
Here is how to use docker-compose to simultaneously start and manage multiple containers (ollama, open-webui, etc.).
Below is an example of a sample docker-compose.yml<\/code> file.<\/p>\nservices:\n ollama:\n image: ollama\/ollama\n container_name: ollama\n ports:\n - \"11434:11434\"\n volumes:\n - .\/ollama_data:\/root\/.ollama\n deploy:\n resources:\n reservations:\n devices:\n - driver: nvidia\n count: all\n capabilities: [gpu]\n\n open-webui:\n image: ghcr.io\/open-webui\/open-webui:main\n container_name: open-webui\n ports:\n - \"3000:8080\"\n volumes:\n - .\/open-webui_data:\/app\/backend\/data\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n depends_on:\n - ollama\n<\/pre>\nTips:<\/em><\/p>\n\n- If you encounter the error message
WARN[0000] \/path\/to\/docker-compose.yml: `version` is obsolete<\/code>, please delete the version line in your docker-compose file. The version line has been omitted in the example above. Apparently, version is no longer required in Docker Compose V2.
\nReference: [Docker Compose] How to resolve the warning `version` is obsolete<\/a> (Japanese)<\/li>\n- If you encounter the error message
Error saving credentials: error storing credentials - err: fork\/exec \/usr\/bin\/docker-credential-desktop.exe: exec format error, out: ``<\/code>, reinstalling Docker Desktop will fix it.
\nReference: Error saving credentials: error storing credentials \u2013 err: fork\/exec \/usr\/bin\/docker-credential-desktop.exe: exec format error, out: \u201c \u00b7 Issue #14039 \u00b7 docker\/for-win<\/a><\/li>\n- Configure
volumes<\/code> to persist data for each container in host folders.<\/li>\n- Use
depends_on<\/code> to explicitly state that open-webui depends on ollama.<\/li>\n- Adjust environment variables and additional settings as needed.<\/li>\n<\/ul>\n
How to Use<\/h2>\n
Enter localhost:3000 in your browser, register an account, and the chat screen will appear.
\n<\/a>It’s telling lies to me…
\n<\/a><\/p>\nIt also looks like the GPU is being utilized.<\/p>\n
References<\/h2>\n\n- Running Gemma3 with Docker in a Local Environment (Ollama+OpenWebUI)<\/a> (Japanese)<\/li>\n
- Setting Up Ollama on Windows Using WSL2 and Docker<\/a> (Japanese)<\/li>\n
- Windows | Docker Docs<\/a><\/li>\n
- Installing the NVIDIA Container Toolkit \u2014 NVIDIA Container Toolkit<\/a><\/li>\n
- How to resolve “The command ‘docker-compose’\uff5e” even when WSL integration is enabled in Docker Desktop<\/a> (Japanese)<\/li>\n
- [Docker Compose] How to resolve the warning `version` is obsolete<\/a> (Japanese)<\/li>\n
- Error saving credentials: error storing credentials \u2013 err: fork\/exec \/usr\/bin\/docker-credential-desktop.exe: exec format error, out: \u201c \u00b7 Issue #14039 \u00b7 docker\/for-win<\/a><\/li>\n<\/ul>\n
Conclusion<\/h2>\n
In this article, we explained how to deploy ollama, open-webui, and gemma3 to a local environment using Docker Desktop (docker-compose) and the NVIDIA Container Toolkit under a WSL2 environment.
We covered everything from installing each tool to managing multiple containers using docker-compose, complete with concrete command and configuration examples.
This makes local LLM operation straightforward, enabling rapid development and secure data management.<\/p>\n
Feel free to customize it to match your own environment and build an efficient LLM setup.<\/p>\n","protected":false},"excerpt":{"rendered":"
In recent years, the demand for LLMs (Large Language Models) has rapidly grown, drawing significant attention […]<\/p>\n","protected":false},"author":4,"featured_media":970,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_locale":"en_US","_original_post":"https:\/\/donguri3.net\/?p=955","footnotes":""},"categories":[1169],"tags":[18,19,51,52,100,365],"class_list":["post-4994","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-local-llm","tag-docker","tag-docker-compose","tag-git","tag-https","tag-ubuntu-22-04","tag-wsl2","en-US"],"_links":{"self":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/4994","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/comments?post=4994"}],"version-history":[{"count":1,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/4994\/revisions"}],"predecessor-version":[{"id":4997,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/4994\/revisions\/4997"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/media\/970"}],"wp:attachment":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/media?parent=4994"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/categories?post=4994"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/tags?post=4994"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Installing WSL2<\/strong> Installing Docker Desktop<\/strong> Installing the NVIDIA Container Toolkit (when using a GPU)<\/strong> Here, we verify the operation of each tool. If you prefer, you can skip ahead to launching via docker-compose.<\/p>\n ollama is a tool for running LLM inference locally. Below is an example of starting a GPU-enabled ollama container.<\/p>\n Tips:<\/em><\/p>\n gemma3 is one of the LLM models compatible with ollama. Tips:<\/em><\/p>\n Here is how to use docker-compose to simultaneously start and manage multiple containers (ollama, open-webui, etc.). Tips:<\/em><\/p>\n Enter localhost:3000 in your browser, register an account, and the chat screen will appear. It also looks like the GPU is being utilized.<\/p>\n In this article, we explained how to deploy ollama, open-webui, and gemma3 to a local environment using Docker Desktop (docker-compose) and the NVIDIA Container Toolkit under a WSL2 environment. Feel free to customize it to match your own environment and build an efficient LLM setup.<\/p>\n","protected":false},"excerpt":{"rendered":" In recent years, the demand for LLMs (Large Language Models) has rapidly grown, drawing significant attention […]<\/p>\n","protected":false},"author":4,"featured_media":970,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_locale":"en_US","_original_post":"https:\/\/donguri3.net\/?p=955","footnotes":""},"categories":[1169],"tags":[18,19,51,52,100,365],"class_list":["post-4994","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-local-llm","tag-docker","tag-docker-compose","tag-git","tag-https","tag-ubuntu-22-04","tag-wsl2","en-US"],"_links":{"self":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/4994","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/comments?post=4994"}],"version-history":[{"count":1,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/4994\/revisions"}],"predecessor-version":[{"id":4997,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/4994\/revisions\/4997"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/media\/970"}],"wp:attachment":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/media?parent=4994"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/categories?post=4994"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/tags?post=4994"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Run the following command in PowerShell (Administrator mode) to install WSL2 and the Ubuntu distribution.<\/p>\nwsl --install -d Ubuntu-24.04<\/pre>\n<\/div>\n<\/div>\n<\/li>\n
Download and install Docker Desktop on Windows from the official website<\/a>, and enable WSL2 integration.
\nIf you encounter the following error when running docker-compose ps<\/code> in your WSL2 environment, try restarting Docker Desktop. For some reason, this fixes it.
\nThe command 'docker-compose' could not be found in this WSL 2 distro. We recommend to activate the WSL integration in Docker Desktop settings. For details about using Docker Desktop with WSL 2, visit: https:\/\/docs.docker.com\/go\/wsl2\/
\n<\/code>Reference: How to resolve “The command ‘docker-compose’\uff5e” even when WSL integration is enabled in Docker Desktop<\/a> (Japanese)<\/p>\n<\/li>\n
Perform the installation on Ubuntu on WSL2 using the following steps.<\/p>\n# Install required packages\nsudo apt-get update && sudo apt-get install -y curl gnupg ca-certificates\n\n# Add NVIDIA's GPG key and repository\ncurl -fsSL https:\/\/nvidia.github.io\/libnvidia-container\/gpgkey | sudo gpg --dearmor -o \/usr\/share\/keyrings\/nvidia-container-toolkit-keyring.gpg\ncurl -s -L https:\/\/nvidia.github.io\/libnvidia-container\/stable\/deb\/nvidia-container-toolkit.list \\\n | sed 's#deb https:\/\/#deb [signed-by=\/usr\/share\/keyrings\/nvidia-container-toolkit-keyring.gpg] https:\/\/#g' \\\n | sudo tee \/etc\/apt\/sources.list.d\/nvidia-container-toolkit.list\nsudo apt-get update\nsudo apt-get install -y nvidia-container-toolkit\n\n# Configure Docker to use the NVIDIA runtime\nsudo nvidia-ctk runtime configure --runtime=docker\nsudo systemctl restart docker\n<\/pre>\n<\/li>\n<\/ol>\n
Installing and Configuring Each Tool (Verification)<\/h2>\n
Introducing ollama<\/h3>\n
Below is an example of starting an ollama container without GPU support.<\/p>\ndocker run -d \\\n -v ollama:\/root\/.ollama \\\n -p 11434:11434 \\\n --name ollama \\\n ollama\/ollama:0.6.2<\/pre>\n
docker run -d --gpus=all<\/span> \\\n -v ollama:\/root\/.ollama \\\n -p 11434:11434 \\\n --name ollama \\\n ollama\/ollama:0.6.2<\/pre>\n\n
-v<\/code> option to persist model data and other files.<\/li>\n--gpus=all<\/code> to configure GPU utilization.<\/li>\nUsing gemma3<\/h3>\n
You can start gemma3 inference by running the following command inside the ollama container.
\ngemma3 is available in model sizes with 1B \/ 4B \/ 12B \/ 27B parameters.
\ngemma3:1b is sufficient for operational verification, so we will use that. Those with abundant PC resources are welcome to try gemma3:4b, gemma3:12b, or gemma3:27b.<\/p>\ndocker exec -it ollama ollama run gemma3:1b<\/pre>\n
\n
OLLAMA_KEEP_ALIVE<\/code>).<\/li>\nStartup Management with docker-compose<\/h2>\n
Below is an example of a sample docker-compose.yml<\/code> file.<\/p>\nservices:\n ollama:\n image: ollama\/ollama\n container_name: ollama\n ports:\n - \"11434:11434\"\n volumes:\n - .\/ollama_data:\/root\/.ollama\n deploy:\n resources:\n reservations:\n devices:\n - driver: nvidia\n count: all\n capabilities: [gpu]\n\n open-webui:\n image: ghcr.io\/open-webui\/open-webui:main\n container_name: open-webui\n ports:\n - \"3000:8080\"\n volumes:\n - .\/open-webui_data:\/app\/backend\/data\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n depends_on:\n - ollama\n<\/pre>\n
\n
WARN[0000] \/path\/to\/docker-compose.yml: `version` is obsolete<\/code>, please delete the version line in your docker-compose file. The version line has been omitted in the example above. Apparently, version is no longer required in Docker Compose V2.
\nReference: [Docker Compose] How to resolve the warning `version` is obsolete<\/a> (Japanese)<\/li>\nError saving credentials: error storing credentials - err: fork\/exec \/usr\/bin\/docker-credential-desktop.exe: exec format error, out: ``<\/code>, reinstalling Docker Desktop will fix it.
\nReference: Error saving credentials: error storing credentials \u2013 err: fork\/exec \/usr\/bin\/docker-credential-desktop.exe: exec format error, out: \u201c \u00b7 Issue #14039 \u00b7 docker\/for-win<\/a><\/li>\nvolumes<\/code> to persist data for each container in host folders.<\/li>\ndepends_on<\/code> to explicitly state that open-webui depends on ollama.<\/li>\nHow to Use<\/h2>\n
\n<\/a>It’s telling lies to me…
\n<\/a><\/p>\nReferences<\/h2>\n
\n
Conclusion<\/h2>\n
We covered everything from installing each tool to managing multiple containers using docker-compose, complete with concrete command and configuration examples.
This makes local LLM operation straightforward, enabling rapid development and secure data management.<\/p>\n