Clustering docker: A guide, with examples....
Basic Manual Cluster
Bash Code For Basic Manual Cluster
Here’s what you’ll need to run docker containers in production in a manual fashion over a cluster of hosts:
Docker Daemon (get.docker.com | bash)
Downloads Docker images
Builds docker images
Runs docker containers
Weave Networking
Gives each container its own internal ip address ex: 10...*
If needed, provides each host with an internatl IP address as well to make connecting system services to those containers easy
Caddy Web server
Automated HTTPS certificates
The "Caddyfile" can be configured to forward a container address like 10.32.0.1 to a web address with a public IP address like www.jimmysfriedchicken.com.
Now, the above platform is manual. It doesn’t automatically assign addresses based on container labels or other factors, and it won’t change the host that containers run on based on system resource availablity. If you want that, you need to know that there are a number of incomplete attempts at achieving such containerization nirvana, such as:
Docker Clustering Systems
Images from: http://www.nextplatform.com/2016/03/29/rancher-rides-herd-containers-borg-style/ (Thanks!)
Docker Swarm (not built into the docker binary)
Docker Swarm Mode 1.12 (built into the docker binary post 1.12)
Kubernetes
Nomad + Consul + Registrator
Rancher
Mesos (to some degree: Mesos is a cluster manager first and foremost, and it is able to use Docker)
Mesosphere (again, to some degree: Mesosphere is Mesos’s Open Source/Commercially Supported big brother that integrates more PaaS like features.)
Kontina
Flynn
Fastest (To get running) Automated Cluster
Docker Swarm Mode
Docker Daemon (get.docker.com | bash)
Downloads Docker images
Builds docker images
Runs docker containers
Swarm Mode
Container networking (won’t need weave)
Host Networking
Scheduling
Orchestration (managing computational load)
But that leaves a significant gap, or depending on your perspective as to what constitutes a complete platform, several signficiant gaps:
Virtual Hosting / Reverse Proxying (though technically the RP/LB is covered, then again, it’s not covered….)
Storage management
Monitoring Service Metrics
Logging Container Logs
But suppose that you could forgeo these items, with the exception of Virtual Hosting. Then, your best bet by far would be to set up Docker-flow-proxy, which mercifully has full support ofr swarm mode since its latest release. So basically docker-flow-proxy is an automated HAPROXY setup that will provide an API to you to call when you launch a container, like this:
From https://github.com/vfarcic/docker-flow-proxy:
Reconfigure
Reconfigures the proxy using information stored in Consul
The following query arguments can be used to send as a reconfigure request to Docker Flow: Proxy. They should be added to the base address :/v1/docker-flow-proxy/reconfigure.
Query | Description | Required | Default | Example |
consulTemplateBePath | The path to the Consul Template representing a snippet of the backend configuration. If specified, the proxy template will be loaded from the specified file. | /consul_templates/tmpl/go-demo-be.tmpl | ||
consulTemplateFePath | The path to the Consul Template representing a snippet of the frontend configuration. If specified, the proxy template will be loaded from the specified file. | /consul_templates/tmpl/go-demo-fe.tmpl | ||
distribute | Whether to distribute a request to all the instances of the proxy. Used only in the swarm mode. | No | false | true |
pathType | The ACL derivative. Defaults to path_beg. SeeHAProxy path for more info. | No | path_beg | |
port | The internal port of a service that should be reconfigured. The port is used only in the swarmmode | Only in swarmmode | ||
serviceDomain | The domain of the service. If specified, the proxy will allow access only to requests coming to that domain. | No | ecme.com | |
serviceName | The name of the service. It must match the name stored in Consul. | Yes | books-ms | |
servicePath | The URL path of the service. Multiple values should be separated by a comma (,). | Yes (unless consulTemplatePath is present) | /api/v1/books | |
skipCheck | Whether to skip adding proxy checks. This option is used only in the defaultmode. | No | false | true |
Remove
Removes a service from the proxy
The following query arguments can be used to send a remove request to Docker Flow: Proxy. They should be added to the base address [PROXY_IP]:[PROXY_PORT]/v1/docker-flow-proxy/remove.
Query | Description | Required | Default | Example |
serviceName | The name of the service. It must match the name stored in Consul | Yes | go-demo | |
distribute | Whether to distribute a request to all the instances of the proxy. Used only in the swarm mode. | No | false | true |
So This is probably the fastest path available today to a functional docker platform. Just use docker swarm mode and then add in docker-flow-proxy. There’s a A LOT that this doesn’t do but that’s kind of the point. If you want to know more about more advanced implementations, just drop me a line at [email protected] preferably using Google Hangouts.
Nice! A bit complicated but still well done!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This is sooooo advanced.. I feel like learning rocket science..
The efforts you've put into this.. Awesome... ^^
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thanks for sharing it. Nice post!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Cool, But... What is Clustering docker?
And if it like the Cloud that you can OWN!
What's the cost of this in ALL of its GLORY!?
Plus this isn't there any more: https://github.com/vfarcic/docker-flow-proxy:
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
There's a lot more than docker-flow-proxy. My setup costs me about $160/mo for four skylake quad core servers with 32gb and 2 SSDs each.
Here's the link to the bash: https://steemit.com/beyondbitcoin/@faddat/systems-geek-series-example-bash-for-roll-your-own-docker-platform-manual
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This post has been linked to from another place on Steem.
Learn more about linkback bot v0.4. Upvote if you want the bot to continue posting linkbacks for your posts. Flag if otherwise.
Built by @ontofractal
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit