The p-pii-checker pipeline checks a deposit’s data files for PII with an LLM. To keep the data away from outside LLM services, the step runs on a machine we control, through a Bitbucket self-hosted runner, and asks the Ollama instance on that machine. This page sets up such a runner. It is done once per machine, by a Bitbucket workspace admin with sudo (or docker group) rights on the machine.
How it fits together¶
The runner is a Docker container on the machine. It polls Bitbucket for steps whose
runs-onlabels it has (self.hosted,linux,pii), and starts each step in a fresh container from the step’s image (python:3.12) on the same machine.The step container reaches Ollama on the host at its default gateway (for example
172.17.0.1:11434), unless the workspace variableOLLAMA_ENDPOINTSsays otherwise.It is a workspace runner, so every
aearep-NNNNrepository can use it.The full results are uploaded to Bitbucket as an artifact of the pipeline run (kept 14 days). The values from the data therefore leave the machine for Bitbucket’s artifact storage, but never for an LLM service.
1. Check the machine¶
Bitbucket asks for a 64-bit Linux host with at least 8 GB of RAM and Docker 19.03 or later. Ollama needs memory for the model as well: gemma4:26b uses about 17 GB, in GPU memory, RAM, or both.
docker version --format '{{.Server.Version}}' # 19.03 or later
free -g # RAM
nvidia-smi # optional: GPUs available to OllamaAtlassian recommends turning swap off (vm.swappiness = 1) on runner hosts, because swapping makes builds slow and unpredictable. On a machine that also serves Ollama and other things, weigh that against those uses.
2. Make Ollama reachable from containers¶
Step containers cannot use localhost to reach the host. Ollama has to listen on an address they can reach:
Ollama in Docker (for example, the
ollamaservice of an Open WebUI stack): publish the port on all interfaces, for exampleports: ["11434:11434"]indocker-compose.yml(not127.0.0.1:11434:11434).Ollama as a system service: set
OLLAMA_HOST=0.0.0.0(sudo systemctl edit ollama, addEnvironment="OLLAMA_HOST=0.0.0.0"under[Service], thensudo systemctl restart ollama).
If a firewall (for example ufw) is active, allow port 11434 from the Docker networks. Then check from a throwaway container:
docker run --rm curlimages/curl -s http://172.17.0.1:11434/api/tags | head -c 300You should see a JSON list of models. If 172.17.0.1 is not the bridge address on this machine, find it with ip -4 addr show docker0.
Pull the model the pipeline uses by default:
ollama pull gemma4:26b # Ollama as a service
docker exec <ollama-container> ollama pull gemma4:26b # Ollama in Docker3. Register the runner in Bitbucket¶
In Bitbucket, open the workspace that holds the
aearep-NNNNrepositories, then the settings cog > Workspace settings > Workspace runners (under Pipelines).Select Add runner.
Under System and architecture, choose Linux Docker (x86_64).
Give it a name that identifies the machine, for example
pii-<hostname>.Under Labels, keep the default labels (
self.hosted,linux) and add the custom labelpii. The pipeline only runs on a runner that has all three.Select Next. The Run step shows a
docker container run ...command. It contains the runner’s OAuth client secret, which Bitbucket shows only once. Copy the whole command now, and treat it like a password.
4. Start the runner¶
Pull the current runner image first:
docker pull docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner:1The command from step 3 looks like this (values elided):
docker container run -it \
-v /tmp:/tmp \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/lib/docker/containers:/var/lib/docker/containers:ro \
-e ACCOUNT_UUID={...} -e RUNNER_UUID={...} \
-e RUNTIME_PREREQUISITES_ENABLED=true \
-e OAUTH_CLIENT_ID=... -e OAUTH_CLIENT_SECRET=... \
-e WORKING_DIRECTORY=/tmp \
--name runner-... \
docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner:1Run it as Bitbucket gave it, with one change: replace -it with -d --restart unless-stopped, so that it runs in the background and comes back after a reboot or a Docker restart. Keep everything else, including the volume mounts.
Then watch the log until the runner reports itself online:
docker logs -f runner-... # should end with the runner reporting itself onlineThe runner’s status in Workspace runners should change to Online.
5. Optional settings¶
OLLAMA_ENDPOINTS: usually not needed. Whichever Docker bridge network a step container is on, its default gateway is the host, so once section 2 is done Ollama is reachable there. Set it only if the step cannot reach Ollama at its default gateway (for example, Ollama on another machine, or a firewall rule). Add a workspace variable (Workspace settings > Workspace variables)OLLAMA_ENDPOINTSwith a URL that works from a container, for examplehttp://192.168.1.4:11434. It applies to every repository.Disk cleanup: the images of past steps (
python:3.12, and older versions of the runner’s own images) stay on the machine. Atlassian suggests pruning Docker on a schedule, butdocker image prune -aremoves every image that no container uses at that moment. On a machine that also runs other stacks, that includes the images of any stack that happens to be down, even locally built ones. Prune only the step images instead: remove images whose name starts withpython:ordocker-public.packages.atlassian.com/sox/atlassian/and that no container (running or stopped) uses. Bitbucket pulls them again when a step needs them. Run that weekly from cron, for example0 3 * * 0 /path/to/prune_step_images.sh >> /path/to/prune.log 2>&1.
6. Test from a branch¶
Create a test repository from the template branch that carries
p-pii-checker(withaeagit-create, giving it the branch), or use an existingaearep-NNNNrepository whosebitbucket-pipelines.ymlhas the pipeline.Make sure
config.ymlnames the deposit (openicpsr,zenodo,dataverseorworldbank), or run1-populate-from-icpsrfirst, which fills it in.p-pii-checkerdownloads the deposit itself.In the repository, go to Pipelines > Run pipeline, pick the branch, choose custom: p-pii-checker, and set
jiraticket(leavePiiModelandPiiCheckerRefat their defaults).Check that:
the step ran on the self-hosted runner (the step details name the runner;
docker pson the machine shows apython:3.12container while it runs);the log shows
run_pii_checker: loading gemma4:26b on http://...followed by pii-checker’s progress;the Jira ticket has a “started” and a “completed” comment with the counts;
the commit added
generated/pii_checker_results-nodetails.xlsxandgenerated/pii_checker_summary.txt;the step’s Artifacts tab offers
pii-checker-full/pii_checker_results.xlsx, and that file is not in the repository.
Troubleshooting¶
| Symptom | Cause and fix |
|---|---|
| The step fails at once, saying no runner matches | No runner with self.hosted, linux and pii is online. Check the runner’s status and docker logs runner-.... |
| The step waits in the queue | The runner is busy with another step. Steps run one at a time per runner. Bitbucket gives up on a queued step after about 2.5 hours. |
could not load gemma4:26b on http://172.17.0.1:11434: ... Connection refused | Ollama is not reachable from containers (section 2), or the gateway is not the host. Set OLLAMA_ENDPOINTS. |
could not load ...: HTTP Error 404 | The model is not pulled. Pull it (section 2) or set PiiModel to a model that is. |
pii-checker did not run: gemma4:26b generated at most ... tokens/s | The model stayed mostly on the CPU for PiiWaitMinutes, because the GPU memory was in use (the message lists other loaded models, for example from Open WebUI). Run the pipeline again once the GPU is free, choose a smaller PiiModel, or lower PiiMinTokensPerSec to accept the slower run. |
The step runs for hours, and llama-server uses many CPU cores | The model is mostly on the CPU. If PiiMinTokensPerSec is 0, nothing stops this: cancel the step, unload the model (docker exec <ollama container> ollama stop gemma4:26b), and run the pipeline again. |
pii-checker did not write ... | pii-checker failed on the package. Its log above the message says why (for example, a .rar archive, or no files). |
| The step stops after 360 minutes | A very large deposit. Raise max-time (up to 720) in bitbucket-pipelines.yml. |
Security notes¶
Any pipeline in the workspace can ask for the
piilabels and run code on this machine, inside containers. From those containers, Ollama and any other service listening on the host or the LAN are reachable.Step containers do not get the host’s Docker socket or file system (a step that asks for the
dockerservice gets a separate docker-in-docker container).To take the runner offline,
docker stop runner-...(anddocker rmit to remove it), or disable or delete it under Workspace runners.
Sources¶
Atlassian: Set up runners for Linux Docker
Atlassian: Adding a new runner in Bitbucket