From 986cd0702d2dfbb8c00d5c1ba2e4e6b7a49d4ea2 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Thu, 12 Dec 2013 09:23:08 -0800 Subject: [PATCH] posts:docker: Add a Docker blog post --- posts/Docker.mdwn | 345 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 345 insertions(+) create mode 100644 posts/Docker.mdwn diff --git a/posts/Docker.mdwn b/posts/Docker.mdwn new file mode 100644 index 0000000..49a7643 --- /dev/null +++ b/posts/Docker.mdwn @@ -0,0 +1,345 @@ +[Docker][] is a tool for managing [Linux containers][wikipedia-lxc] +([LXC][lxc]). LXC allows you to provision multiple, isolated +process-spaces on the same kernel, which is useful for creating +disposable development/staging/production environments. Docker +provides a sleek frontend for container managerment. Like [[Salt +Stack|Salt_Stack]], it uses a daemonized manager with a command-line +client. + +Installation +============ + +To avoid the [aufs3][] dependency, I wanted [Docker v0.7+][v0.7]. The +main portage tree is currently at Docker v0.6.6, so I installed the +[docker overlay][overlay]: + + # layman -a docker + +I also wanted Go v1.2+ to avoid a [“slice bounds out of range” on +/usr/lib/go/src/pkg/archive/tar/writer.go:233][1509] (fixed upstream +in [0c7e4c4, archive/tar: Fix support for long links and improve PAX +support, 2013-08-18][0c7e4c4]. Thanks to [Jonathan Stoppani][JS] for +[help figuring this out][overlay-11]). That version is not in the +main Portage tree yet, but [it is in][go-zugaina] the [OSSDL +overlay][OSSDL]: + + # layman -a OSSDL + +After that, [installing Docker][gentoo] on [[Gentoo]] is the usual: + + # emerge -av app-emulation/docker + +After reading through a few docs about IP forwarding, I determined +that my internal development box could safely enable IP forwarding, +which containers use to connect to the outside network. I enabled it +for subsequent reboots: + + # echo net.ipv4.ip_forward = 1 > /etc/sysctl.d/docker.conf + +I also had to enable the following additional kernel options (as +suggested by `app-emulation/lxc`): `CONFIG_CGROUP_DEVICE`, +`CONFIG_USER_NS`, `CONFIG_DEVPTS_MULTIPLE_INSTANCES`, `CONFIG_VETH`, +and `CONFIG_MACVLAN`. There's also the convenient `lxc-checkconfig` +script distributed with [app-emulation/lxc][lxc], which pointed out +the need for `CONFIG_CGROUP_MEM_RES_CTLR` (renamed to `CONFIG_MEMCG` +in [c255a45 (memcg: rename config variables, 2012-07-31)][c255a45], +released in Linux v3.6) and `CONFIG_VLAN_8021Q`. On top of those, +`app-emulation/docker` recommended `CONFIG_BRIDGE`, +`CONFIG_NETFILTER_XT_MATCH_ADDRTYPE`, `CONFIG_NF_NAT`, +`CONFIG_NF_NAT_NEEDED`, `CONFIG_IP_NF_TARGET_MASQUERADE` (since +[045eb9f (another necessary kernel flag, 2013-12-09)][045eb9f], in a +fast response to my [comment][866-wtk]), and +`CONFIG_DM_THIN_PROVISIONING`. These are the new docker-supporting +lines in my `.config` for Linux v3.10: + + CONFIG_CGROUP_DEVICE=y + CONFIG_MEMCG=y + CONFIG_USER_NS=y + CONFIG_UIDGID_STRICT_TYPE_CHECKS=y + CONFIG_MM_OWNER=y + CONFIG_NETFILTER_ADVANCED=y + CONFIG_BRIDGE_NETFILTER=y + CONFIG_NF_NAT=m + CONFIG_NF_NAT_NEEDED=y + CONFIG_NF_NAT_FTP=m + CONFIG_NF_NAT_IRC=m + CONFIG_NF_NAT_SIP=m + CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m + CONFIG_NF_NAT_IPV4=m + CONFIG_IP_NF_TARGET_MASQUERADE=m + CONFIG_NF_NAT_IPV6=m + CONFIG_IP6_NF_TARGET_MASQUERADE=m + CONFIG_STP=m + CONFIG_BRIDGE=m + CONFIG_BRIDGE_IGMP_SNOOPING=y + CONFIG_VLAN_8021Q=m + CONFIG_LLC=m + CONFIG_DM_BUFIO=m + CONFIG_DM_BIO_PRISON=m + CONFIG_DM_PERSISTENT_DATA=m + CONFIG_DM_THIN_PROVISIONING=m + CONFIG_MACVLAN=m + CONFIG_VETH=m + CONFIG_DEVPTS_MULTIPLE_INSTANCES=y + +After that, I rebuilt my kernel, rebooted into the new kernel, and +started the daemon with: + + # /etc/init.d/docker start + +Command line usage +================== + +Docker does a good job with interactive help: + + $ docker + Usage: Docker [OPTIONS] COMMAND [arg...] + -H="127.0.0.1:4243": Host:port to bind/connect to + + A self-sufficient runtime for linux containers. + + Commands: + + attach Attach to a running container + build Build a container from a Dockerfile + commit Create a new image from a container's changes + … + +Docker images are [archived][] for easy reuse, so it's likely someone +has the image you need (or a good start) already built. Docker's +images consist of a number of layers on top of the base image: + + $ docker pull learn/tutorial + Pulling repository learn/tutorial from https://index.docker.io/v1 + Pulling image 8dbd9e392a964056420e5d58ca5cc376ef18e2de93b5cc90e868a1bbc8318c1c (precise) from ubuntu + Pulling image b750fe79269d2ec9a3c593ef05b4332b1d1a02a62b4accb2c21d589ff2f5f2dc (12.10) from ubuntu + Pulling image 27cf784147099545 () from tutorial + +The image has everything needed by a process (filesystem, system +libraries, …), so you spin up the container by specifiying an image +and target command. + + $ docker run learn/tutorial echo hello world + hello world + +Changes made to a container are preserved between runs: + + $ docker run learn/tutorial apt-get install -y ping + +You can checkpoint changes by committing, which adds a new layer: + + $ docker ps -l + ID IMAGE COMMAND CREATED STATUS PORTS + 6982a9948422 ubuntu:12.04 apt-get install ping 1 minute ago Exit 0 + $ docker commit 6982a99 learn/ping + effb66b31edb + $ docker run learn/ping ping www.google.com + 64 bytes from nuq05s02-in-f20.1e100.net (74.125.239.148): icmp_req=1 ttl=55 time=2.23 ms + 64 bytes from nuq05s02-in-f20.1e100.net (74.125.239.148): icmp_req=2 ttl=55 time=2.30 ms + ^C + +You can list running containers: + + $ docker ps + ID IMAGE COMMAND CREATED STATUS PORTS + efefdc74a1d5 learn/ping:latest ping www.google.com 37 seconds ago Up 36 seconds + $ docker inspect efefdc7 + [2013/07/30 01:52:26 GET /v1.3/containers/efef/json + { + "ID": "efefdc74a1d5900d7d7a74740e5261c09f5f42b6dae58ded6a1fde1cde7f4ac5", + "Created": "2013-07-30T00:54:12.417119736Z", + "Path": "ping", + "Args": [ + "www.google.com" + ], + "Config": { + "Hostname": "efefdc74a1d5", + "User": "", + "Memory": 0, + "MemorySwap": 0, + "CpuShares": 0, + "AttachStdin": false, + "AttachStdout": true, + "AttachStderr": true, + "PortSpecs": null, + "Tty": false, + "OpenStdin": false, + "StdinOnce": false, + "Env": null, + "Cmd": [ + "ping", + "www.google.com" + ], + "Dns": null, + "Image": "learn/ping", + "Volumes": null, + "VolumesFrom": "", + "Entrypoint": null + }, + "State": { + "Running": true, + "Pid": 22249, + "ExitCode": 0, + "StartedAt": "2013-07-30T00:54:12.424817715Z", + "Ghost": false + }, + "Image": "a1dbb48ce764c6651f5af98b46ed052a5f751233d731b645a6c57f91a4cb7158", + "NetworkSettings": { + "IPAddress": "172.16.42.6", + "IPPrefixLen": 24, + "Gateway": "172.16.42.1", + "Bridge": "docker0", + "PortMapping": { + "Tcp": {}, + "Udp": {} + } + }, + "SysInitPath": "/usr/bin/docker", + "ResolvConfPath": "/etc/resolv.conf", + "Volumes": {}, + "VolumesRW": {} + } + +And you can push your local images to the repository: + + $ docker images + ubuntu latest 8dbd9e392a96 4 months ago 131.5 MB (virtual 131.5 MB) + learn/tutorial latest 8dbd9e392a96 2 months ago 131.5 MB (virtual 131.5 MB) + learn/ping latest effb66b31edb 10 minutes ago 11.57 MB (virtual 143.1 MB) + $ docker push learn/ping + +You can also run interactive shells: + + $ docker pull ubuntu + $ docker run -i -t ubuntu /bin/bash + +You can also [set environment variables][run], which are useful for +customizing generic images: + + $ docker run -e HOST=example.net -e PORT=1234 ubuntu + +Dockerfiles +=========== + +Instead of building containers manually, you can also specify them +with a [Dockerfile][] ([spec][Dockerfile-spec]): + + $ cat gentoo-portage/Dockerfile + FROM tianon/gentoo + MAINTAINER W. Trevor King, wking@tremily.us + RUN echo 'GENTOO_MIRRORS="http://mirror.mcs.anl.gov/pub/gentoo/"' >> /etc/portage/make.conf + #RUN echo 'SYNC="rsync://rsync.us.gentoo.org"' >> /etc/portage/make.conf + RUN mkdir -p /usr/portage + RUN emerge-webrsync + RUN emerge --sync --quiet + RUN eselect news read new + $ docker build -t wking/gentoo-portage gentoo-portage + … + $ cat gentoo-portage-en-us/Dockerfile + FROM wking/gentoo-portage + MAINTAINER W. Trevor King, wking@tremily.us + RUN echo en_US ISO-8859-1 > /etc/locale.gen + RUN echo en_US.UTF-8 UTF-8 >> /etc/locale.gen + RUN locale-gen + RUN echo 'LANG="en_US.UTF-8"' >> /etc/env.d/02locale + RUN env-update + $ docker build -t wking/gentoo-portage-en-us gentoo-portage-en-us + $ cat gentoo-portage-en-us-syslog/Dockerfile + FROM wking/gentoo-portage-en-us + MAINTAINER W. Trevor King, wking@tremily.us + RUN emerge -v sys-process/vixie-cron app-admin/syslog-ng app-admin/logrotate + RUN rc-update add syslog-ng default + RUN rc-update add vixie-cron default + $ docker build -t wking/gentoo-portage-en-us-syslog gentoo-portage-en-us-syslog + +You can't currently set the description (returned by `docker search +…`) from the `Dockerfile`, although there [are][769] [some][2167] +[proposals][2956] to add this feature. + +You can mount volumes from the `Dockerfile` using [VOLUME][], but it +[doesn't][1124-explicit] support host mounts. For those you have to +use the [-v][run-v] option when you invoke `docker run …`: + + $ docker run -i -t -v /usr/portage:/usr/portage:ro -v /usr/portage/distfiles:/usr/portage/distfiles:rw wking/gentoo-portage /bin/bash + +Bridging +======== + +Docker sets up a `docker0` bridge between the host's network and the +containers. Docker tries to guess an IP range that does not conflict +with your local network, but [it's not omniscient][1558]. Until +[1558][] is fixed, you're best bet is to set up your own bridge. If +you already have a `docker0` bridge: + + $ ip addr show dev docker0 + 5: docker0: mtu 1500 qdisc noqueue state DOWN + link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff + inet 10.0.42.1/16 scope global docker0 + valid_lft forever preferred_lft forever + … + +then you can just change it's address: + + # ip addr del 10.0.42.1/16 dev docker0 + # ip addr add 172.31.42.1/16 dev docker0 + +If the `docker0` bridge doesn't already exist, you'll have to [create +it yourself][bridge]: + + # brctl addbr docker0 + # ip addr add 172.31.0.1/16 dev docker0 + # ip link set dev docker0 up + +If you want to start over from scratch, you can stop docker and remove +the bridge: + + # /etc/init.d/docker stop + # ip link set dev docker0 down + # brctl delbr docker0 + +Linking +======= + +You can [link containers by name][linking]: + + $ docker run -d -name redis crosbymichael/redis + $ sudo docker run -link redis:db -d -name webapp me/someapp + +This links `webapp` to `redis` using the alias `db`. `DB_PORT` (and +similar) environment variables are set in the `webapp` container, +which it can use to connect to `redis` container. You can also use +[ambassadors][] to link containers on separate hosts. + +[Docker]: http://www.docker.io/ +[wikipedia-lxc]: http://en.wikipedia.org/wiki/LXC +[1509]: https://github.com/dotcloud/docker/pull/1509 +[0c7e4c4]: http://code.google.com/p/go/source/detail?r=0c7e4c45acf8 +[JS]: https://github.com/GaretJax +[overlay-11]: https://github.com/tianon/docker-overlay/issues/11 +[go-zugaina]: http://gpo.zugaina.org/dev-lang/go +[OSSDL]: https://mark.ossdl.de/tag/ossdl-overlay/ +[gentoo]: http://docs.docker.io/en/latest/installation/gentoolinux/ +[aufs3]: http://aufs.sourceforge.net/ +[v0.7]: http://blog.docker.io/2013/11/docker-0-7-docker-now-runs-on-any-linux-distribution/ +[overlay]: https://github.com/tianon/docker-overlay/ +[lxc]: http://lxc.sourceforge.net/ +[c255a45]: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c255a458055e459f65eb7b7f51dc5dbdd0caf1d8 +[045eb9f]: https://github.com/tianon/docker-overlay/commit/045eb9f54bbfad07657ca1dafeb84d56c56fdbea +[866-wtk]: https://github.com/dotcloud/docker/issues/866#issuecomment-30190904 +[archived]: https://index.docker.io/ +[run]: http://docs.docker.io/en/latest/commandline/cli/#run +[Dockerfile]: http://www.docker.io/learn/dockerfile/ +[Dockerfile-spec]: http://docs.docker.io/en/latest/use/builder/ +[769]: https://github.com/dotcloud/docker/issues/769 +[2167]: https://github.com/dotcloud/docker/issues/2176 +[2956]: https://github.com/dotcloud/docker/pull/2956 +[VOLUME]: http://docs.docker.io/en/latest/use/builder/#volume +[1124-explicit]: https://github.com/dotcloud/docker/pull/1124#issuecomment-22824230 +[run-v]: http://docs.docker.io/en/latest/use/working_with_volumes/ +[1558]: https://github.com/dotcloud/docker/issues/1558 +[bridge]: http://docs.docker.io/en/latest/use/networking/ +[linking]: http://docs.docker.io/en/latest/use/working_with_links_names/ +[ambassadors]: http://docs.docker.io/en/latest/use/ambassador_pattern_linking/ + +[[!tag tags/linux]] +[[!tag tags/tools]] -- 2.26.2