最新消息:比度技术-是关注云计算、大数据、分布式存储、高并发、高性能、人工智能等互联网技术的个人博客。

Docker跨主机通讯-etcd服务发现

云计算 bidu 755浏览

Docker跨主机通讯-etcd服务发现

docker实现多主机互联有很多方法,基于openvswitch等。docker在1.9之后支持overlay network特性。可以方便的实现多主机互联。用

docker创建overlay网络,需要一个key-value存储服务,存储各个主机节点在overlay网络中的配置信息

[root@07Node ~]# vim /lib/systemd/system/docker.service

添加etcd存储 ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock –cluster-

advertise=172.16.40.7:2375 –cluster-store=etcd://172.16.40.7:2379,172.16.10.216:2379,172.16.10.219:2379/netoverlaypool —

storage-driver=devicemapper

通过swarm查看docker 集群网络信息



[root@219Node ~]# docker -H 172.16.40.7:2376 network ls
NETWORK ID          NAME                      DRIVER
099c353db598        07Node/bridge             bridge              
6ff9e687037b        07Node/docker_gwbridge    bridge              
3f3c7ce382e6        07Node/host               host                
ee80a159a083        07Node/none               null                
3ccdfbf2171e        216Node/bridge            bridge              
14a0baa95b49        216Node/docker_gwbridge   bridge              
3c045376f0d2        216Node/host              host                
1520ebe259db        216Node/none              null                
5b887c89e45c        219Node/bridge            bridge              
4e0862f9e2de        219Node/docker_gwbridge   bridge              
1736c72bcec3        219Node/host              host                
b5fcb8f0a3a7        219Node/none              null                
169d5c9b0ed9        overlaynetCluster         overlay             
c06ce0837fd3        overlaynetCluster2        overlay  


[root@219Node ~]# docker -H 172.16.40.7:2376 network inspect overlaynetCluster
[
    {
        "Name": "overlaynetCluster",
        "Id": "169d5c9b0ed9d1ae8ab0b1944f29752b0d1578f9c0e8bb74ac730d1dece7034d",
        "Scope": "global",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "10.0.1.0/24",
                    "Gateway": "10.0.1.1/24"
                }
            ]
        },
        "Internal": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]
[root@219Node ~]# docker -H 172.16.40.7:2376 network inspect overlaynetCluster2
[
    {
        "Name": "overlaynetCluster2",
        "Id": "c06ce0837fd3ebe4a206e6d096023014e0cda1105af41cff6430fb07b2d9d4e8",
        "Scope": "global",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "10.0.2.0/24",
                    "Gateway": "10.0.2.1/24"
                }
            ]
        },
        "Internal": false,
        "Containers": {
            "0ef1aaa110ab521ba4eebd92e623255ce11056ddba50447fdf61931337af0371": {
                "Name": "ubantu005",
                "EndpointID": "245a6871896970e22df6b40c60a5f0ca8b22ed5cae7274f41d837739a216bbd6",
                "MacAddress": "02:42:0a:00:02:06",
                "IPv4Address": "10.0.2.6/24",
                "IPv6Address": ""
            },
            "196ffe88f4e5924fb09fc06f6f73e50b5b5a14e455b6d11c439eec228fd6a34a": {
                "Name": "ubantu003",
                "EndpointID": "9dc50b19d6a49922bc8913504e80a81d28ed5df1e1313740c352c1c20fb76804",
                "MacAddress": "02:42:0a:00:02:04",
                "IPv4Address": "10.0.2.4/24",
                "IPv6Address": ""
            },
            "1ed18a78b6191040d9af06d98c003c0136f829ec898db27b6ae53d36a2cb9214": {
                "Name": "ubantu015",
                "EndpointID": "e6a623c39b02c8f072f90a4020b3807b5749821cb6dc79d21aa0724c8f826aeb",
                "MacAddress": "02:42:0a:00:02:10",
                "IPv4Address": "10.0.2.16/24",
                "IPv6Address": ""
            },
            "2389dc5355d63b556ee5e5051de5d3389ddf91a3368145576d629206f9ce3f3f": {
                "Name": "ubantu001",
                "EndpointID": "6aa311e4367e97276f73b4caca92697649b28f9285aac0e5bd685611ac6ea885",
                "MacAddress": "02:42:0a:00:02:02",
                "IPv4Address": "10.0.2.2/24",
                "IPv6Address": ""
            },
            "30c213de02449931595c3a68b2857965a07f12acd2ded171d67c5163b0b3bd5f": {
                "Name": "ubantu010",
                "EndpointID": "730c699f696d4fa8928bab2253c89bfbaeb3b28c5421be632309c5679563e137",
                "MacAddress": "02:42:0a:00:02:0b",
                "IPv4Address": "10.0.2.11/24",
                "IPv6Address": ""
            },
            "324013a48aed67117e84305cda7c9d1c208964f0ceb33f7c59354affea0de90a": {
                "Name": "ubantu014",
                "EndpointID": "3635117eddb0f4bc7403f2ffa3c7e695cfbd8f113111222bd3783e2fe8213bc5",
                "MacAddress": "02:42:0a:00:02:0f",
                "IPv4Address": "10.0.2.15/24",
                "IPv6Address": ""
            },
            "348ad665e8125e5bd0b19c1dd0ade234669bc822b0cfca1ba61c64615a9e6364": {
                "Name": "ubantu007",
                "EndpointID": "8070f918b1d9a162ebbad6dcc2d53f603934378b47c8ab1a6a25b9340f53aadd",
                "MacAddress": "02:42:0a:00:02:08",
                "IPv4Address": "10.0.2.8/24",
                "IPv6Address": ""
            },
            "34c14c86b1b5026cf7cda35c1613973b2ed9992f1741393b1bda77fad969b77c": {
                "Name": "ubantu004",
                "EndpointID": "b3e5b41bb66d66ccace839da0cbcb5d870460ae2284da26720d34dced5e632b1",
                "MacAddress": "02:42:0a:00:02:05",
                "IPv4Address": "10.0.2.5/24",
                "IPv6Address": ""
            },
            "6e3391eb80a6884447249490ab693749a3bbef71dabc3d6cb240904c086c2447": {
                "Name": "ubantu008",
                "EndpointID": "f1aa46b3a5c7f3daf20cb121d76aae4107fe6218fa78b0201505753f726b2a3b",
                "MacAddress": "02:42:0a:00:02:09",
                "IPv4Address": "10.0.2.9/24",
                "IPv6Address": ""
            },
            "6effb3ba4b34336ef890bebf9e63e636af13f92fb13de43350933848064cfa1f": {
                "Name": "ubantu009",
                "EndpointID": "2718d39461422f4133ecce0439120e8122830c544a3090c6c3b3dd201676f8ee",
                "MacAddress": "02:42:0a:00:02:0a",
                "IPv4Address": "10.0.2.10/24",
                "IPv6Address": ""
            },
            "74df950e6551a625af13140ad8091f21af2c296983d2b2628c663f827976ec77": {
                "Name": "ubantu012",
                "EndpointID": "900f52ba1f79c01d4f351878ff76cfddba292d88a7e1cb4af2a11ace5f648127",
                "MacAddress": "02:42:0a:00:02:0d",
                "IPv4Address": "10.0.2.13/24",
                "IPv6Address": ""
            },
            "7773a0be0b6bad67030c6a609933464194d73def38365b016f63bcab0e787479": {
                "Name": "ubantu011",
                "EndpointID": "7c827778983ced96cb2ba34d1169edca37aa13c3561780f6a8e60429a2145cb6",
                "MacAddress": "02:42:0a:00:02:0c",
                "IPv4Address": "10.0.2.12/24",
                "IPv6Address": ""
            },
            "7b060eafff9cbe3dc06fde41cf6f1b2b58e01074d17cc4455e8c5ea3b0334527": {
                "Name": "ubantu013",
                "EndpointID": "5df5d29c9b6a1dde302c05c8b0ed5af9839fe5ee8f825437fbd3c46568b872e7",
                "MacAddress": "02:42:0a:00:02:0e",
                "IPv4Address": "10.0.2.14/24",
                "IPv6Address": ""
            },
            "efee35af8c3e0a4f4fab28d05937242b7370b2925a6b844b6e73793c789f3c53": {
                "Name": "ubantu006",
                "EndpointID": "63771d1f8298da3869bf12375ac9567945f7b5c757b75e539df7be22a92ba775",
                "MacAddress": "02:42:0a:00:02:07",
                "IPv4Address": "10.0.2.7/24",
                "IPv6Address": ""
            },
            "fd04e404ddc5603d64a37a345acb6d34f42fc22f7189c1570f4bb4209ce0bd33": {
                "Name": "ubantu002",
                "EndpointID": "1ab6a17d60c7aeef9251787198e47f0a2dc1d85a4cc249c7a14b9fce41f67984",
                "MacAddress": "02:42:0a:00:02:03",
                "IPv4Address": "10.0.2.3/24",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]
[root@219Node ~]# 

etcd里保存着整个overlay网络配置和节点信息

etcd存储的内容:

参考:etcd client api curl http://172.16.10.219:2379/v2/keys


{
    "action": "get",
    "node": {
        "dir": true,
        "nodes": [
            {
                "key": "/docker",
                "dir": true,
                "modifiedIndex": 861,
                "createdIndex": 861
            },
            {
                "key": "/message",
                "value": "cluser Hello world",
                "modifiedIndex": 10,
                "createdIndex": 10
            },
            {
                "key": "/no",
                "dir": true,
                "modifiedIndex": 119,
                "createdIndex": 119
            },
            {
                "key": "/nodes",
                "dir": true,
                "modifiedIndex": 14,
                "createdIndex": 14
            }
        ]
    }
}

curl http://172.16.10.219:2379/v2/keys/docker/network



{
    "action": "get",
    "node": {
        "key": "/docker/network",
        "dir": true,
        "nodes": [
            {
                "key": "/docker/network/v1.0",
                "dir": true,
                "modifiedIndex": 862,
                "createdIndex": 862
            }
        ],
        "modifiedIndex": 862,
        "createdIndex": 862
    }
}

curl http://172.16.10.219:2379/v2/keys/docker/network/v1.0/overlay/network/


{
    "action": "get",
    "node": {
        "key": "/docker/network/v1.0/overlay/network",
        "dir": true,
        "nodes": [
            {
                "key": "/docker/network/v1.0/overlay/network/c06ce0837fd3ebe4a206e6d096023014e0cda1105af41cff6430fb07b2d9d4e8",
                "value": "[{\"SubnetIP\":\"10.0.2.0/24\",\"GwIP\":\"10.0.2.1/24\",\"Vni\":257}]",
                "modifiedIndex": 26810,
                "createdIndex": 26429
            },
            {
                "key": "/docker/network/v1.0/overlay/network/169d5c9b0ed9d1ae8ab0b1944f29752b0d1578f9c0e8bb74ac730d1dece7034d",
                "value": "[{\"SubnetIP\":\"10.0.1.0/24\",\"GwIP\":\"10.0.1.1/24\",\"Vni\":256}]",
                "modifiedIndex": 26507,
                "createdIndex": 1979
            }
        ],
        "modifiedIndex": 1953,
        "createdIndex": 1953
    }
}


curl http://172.16.10.219:2379/v2/keys/docker/network/v1.0/overlay/network/c06ce0837fd3ebe4a206e6d096023014e0cda1105af41cff6430fb07b2d9d4e8/

{
    "action": "get",
    "node": {
        "key": "/docker/network/v1.0/overlay/network/c06ce0837fd3ebe4a206e6d096023014e0cda1105af41cff6430fb07b2d9d4e8",
        "value": "[{\"SubnetIP\":\"10.0.2.0/24\",\"GwIP\":\"10.0.2.1/24\",\"Vni\":257}]",
        "modifiedIndex": 26810,
        "createdIndex": 26429
    }
}

与:docker -H 172.16.40.7:2376 network inspect overlaynetCluster2
结果一致

swarm 集群:

curl http://172.16.10.219:2379/v2/keys/docker/nodes



{
    "action": "get",
    "node": {
        "key": "/docker/nodes",
        "dir": true,
        "nodes": [
            {
                "key": "/docker/nodes/172.16.10.216:2375",
                "value": "172.16.10.216:2375",
                "expiration": "2016-05-23T06:36:33.165585707Z",
                "ttl": 51,
                "modifiedIndex": 95346,
                "createdIndex": 95346
            },
            {
                "key": "/docker/nodes/172.16.10.219:2375",
                "value": "172.16.10.219:2375",
                "expiration": "2016-05-23T06:36:39.24977349Z",
                "ttl": 57,
                "modifiedIndex": 95349,
                "createdIndex": 95349
            },
            {
                "key": "/docker/nodes/172.16.40.7:2375",
                "value": "172.16.40.7:2375",
                "expiration": "2016-05-23T06:36:35.227859196Z",
                "ttl": 53,
                "modifiedIndex": 95347,
                "createdIndex": 95347
            }
        ],
        "modifiedIndex": 861,
        "createdIndex": 861
    }
}


###Swarm集群高可用-swarm manager主从节点

参考:http://www.liyuduo.com/?p=3091



etcd 参考:
https://coreos.com/etcd/docs/latest/api.html

转载请注明:比度技术-关注互联网技术的个人博客 » Docker跨主机通讯-etcd服务发现