1 Docker

Notes following full stack open’s docker chapter.

Basic commands

Definition 1.1 (containers and images) A docker image is an immutable file (collection). A container is a running (and mutable) instance of an image. Container can be crystallized into containers.

Running containers: docker container run hello-world
- Interactive mode: docker container run -it ubuntu bash.
- docker container run [OPTIONS] IMAGE [COMMAND]
- Remove after execution: docker container run --rm ubuntu ls .
- Run with name: docker container run -it --name [name] [doncainter name] [command]
- Run with binding: docker container run ... -v FILE-IN-HOST:FILE-IN-CONTAINER. Binds will be visible on both sides.
Image ops:
- Listing: docker image ls
Container operations:
- Listing: docker container ls -a (includes exited ones)
- Deleting: docker container rm [ID prefix].
- Starting an initialized container: docker start [container name]
  - Start with \(-it\) to create a binding, interactive process
- Kill main process of running container: docker kill [container]
- Create additional connection: docker exec -it [name] /bin/bash.
Committing & manipulation:
- docker commit CONTAINER-ID-OR-CONTAINER-NAME NEW-IMAGE-NAME
- Copy: docker container cp [local path] [image-name]:[container path]

Sample docker image: pytorch/pytorch:2.7.0-cuda11.8-cudnn9-devel

Sample workflow


docker container ls -a 
docker image ls -a 

docker container run -it --gpus all --name gpu_starter pytorch/pytorch:2.7.0-cuda11.8-cudnn9-devel

docker start gpu_starter
docker attach 
docker exec -it gpu_starter /bin/bash
docker kill gpu_starter 

docker container cp 01-docker.Rmd  gpu_starter:/workspace

Dockerfile

Dockerfile defines the workflow for building an image.

Sample dockerfile:

FROM pytorch/pytorch:2.7.0-cuda11.8-cudnn9-devel
COPY ./simple.py /workspace 
CMD python /workspace/simple.py

Where simple.py

Run in the dockerfile folder: docker build . -t gpu_starter
Running: docker container run --gpus all --name gpu -p 8888:8888 --rm -it gpu_starter

1.1 Docker compose

Note that the workflow above still involves (1) using the Dockerfile to build an image, and (2) running the image with specified parameters. The second can be automated by docker compose. Equivalent specification in docker-compose.yml:

services:
  app: 
    image: pytorch/pytorch:2.7.0-cuda11.8-cudnn9-devel
    build: .
    stdin_open: true 
    tty: true 
    ports:
      - 8888:8888
    gpus: all 
    volumes: 
      - ~/Downloads:/workspace/downloads

Running docker compose up --build will create the image and
- Images are retained. Remove by docker compose down --rmi all
- Use docker compose up --build -d to detach after building
Compose up will also automatically create containers. Use docker compose down to remove all containers for compose services.
Run docker container ls -a and docker attach [container] to interact

Publishing images

To publish, run

docker login
docker tag sample_image:latest nlyu1/sample_image:latest
docker push nlyu1/sample_image:latest

To pull from elsewhere, run docker pull nlyu1/sample_image:latest.

Explanation from ChatGPT

When you run docker compose up and down, Compose is really orchestrating the same Docker primitives you know from docker build, docker run, docker stop, and docker rm—but it does it for all your services in one go.

1.1.1 1. Which Dockerfile is used? Do I see the build logs?

Build context & Dockerfile location Because you have
```
build: .
```
Compose will look in . (your project root) for a file named Dockerfile.
Image tag You also set
```
image: hiyouga/llamafactory:latest
```
so after it builds that Dockerfile, it tags the resulting image as hiyouga/llamafactory:latest.
Logs
- If the image doesn’t already exist locally (or if you pass --build), Compose will invoke docker build . for you and stream the usual build logs right into your terminal.
- On subsequent up calls, if nothing has changed in the build context, Compose will skip the rebuild unless you force it.

2. What happens when you run `docker compose up`?

Networks & volumes Compose creates any networks and named volumes declared in your file (if they don’t already exist).
Images
- For services with a build: block, it builds (or reuses) the image.
- For services without build:, it pulls from registry if missing.
Containers
- If no container exists for llamafactory, it creates one (same as docker create …) and then starts it.
- If a container has already been created by Compose:
  - Unchanged config: it simply starts the stopped container (so you keep your previous data in volumes, etc.).
  - Changed config or forced recreate: Compose will tear down the old container and spin up a fresh one.
Flags you recognize
- stdin_open: true + tty: true → the -it behavior
- ipc: host, gpus: all, ports:, volumes: → passed straight through under the hood

You can always run

docker compose up --build --force-recreate

to rebuild images and recreate containers every time.

3. What happens when you run `docker compose down`?

Stops all running containers in that Compose project (same as docker stop).
Removes those containers and the default network that Compose created.
Leaves behind:
- The images you built or pulled.
- Any named volumes (so your data survives) unless you add -v.
If you want to clean everything up, you can do:
```
docker compose down --rmi all -v
```
which also removes images (--rmi all) and named volumes (-v).

TL;DR Workflow

Write your Dockerfile in .
Declare in docker-compose.yml both build: . and image: your/name:tag if you want a custom build and a consistent tag
docker compose up
- builds (if needed),
- creates (if needed),
- starts (or restarts) your containers
docker compose down
- stops & removes containers and network,
- leaves images & volumes unless you tell it otherwise

That’s the Compose lifecycle mapping back to the classic Docker commands you already know.