Create networks automatically in Docker Compose

Question

When using custom networks in Docker like

networks:
  default:
    external:
      name: service-tier

And try to run that, I´m getting the error

ERROR: Network service-tier declared as external, but could not be found. Please create the network manually using docker network create service-tier and try again.

Is it possible to create networks automatically with docker-compose or is there no other way than creating them manually first?

Grimmy · Accepted Answer · 2017-07-22T20:53:16.333

48

external is to use an existing network. If you want compose to make networks you simply do:

networks:
  network1:
  network2:

.. to instruct compose to make the networks. They will be named <compose-dir>-<network name> Verify the creation by using docker network ls.

You can also override the default network or specify more properties.

networks:
  default:
    driver: bridge
    driver_opts:
      com.docker.network.driver.mtu: 1450

.. will override the default network setting MTU to 1450 (handy when the host have lower than 1500 mtu and path mtu discovery doesn't work properly). There are other bridge driver options.

external makes more sense when you want services in two compose setups to talk to each other or you have a manually created network.

edited Jul 22 '17 at 20:53

answered Jul 22 '17 at 15:16

Grimmy

3,992
22
25

how do i check for driver options other than bridge? specifying com.docker.network.driver.mtu works for bridge driver, but not macvlan driver – ealeon Oct 01 '18 at 06:45
@ealeon I'm guessing the only place to look for that is the driver docs – Grimmy Oct 16 '18 at 07:53
2

@Grimmy Is possible to create persistent networks on docker compose up? Cause what is happening to me is that when i shut down the containers, the docker compose is also removing the network. – Gonzalo May 20 '19 at 18:21
That last sentence is worth a hundred upvotes. People often leave out the vital "and is generally used in X scenario" so we know if it's what we're looking for when unfamiliar with something. This perfectly confirms the use case. – James Jul 27 '21 at 23:27
I do need two compose setups to talk to each others and that's why I use `external`. But I wish docker compose could create this network itself if it doesn't exist. The setups are already complex without develper having to manually precreate networks – Gherman Apr 19 '23 at 14:44

helvete · Answer 2 · 2018-03-02T07:53:02.143

41

As @Grimmy correctly states, docker creates all mentioned networks, so these can be later referenced by running another compose file.

But, the defaultly generated network name is hardly practlical or robust. As it could turn out to be very long or the docker team changes their opinion on naming strategy.

But since compose file version 3.5 (means docker-compose version 1.18.0) one can name the network as pleases, so the overal solution is even more robust.

Please see the following snippets demonstrating on how to achieve this:

Compose file 1

version: '3.5'
services:
  svc-name:

    ...                          

    networks:
      - specific-network-name
    container_name: "exact-container-reference"

    ...

networks:
  specific-network-name:
    external: false
    name: specific-network-name

Compose file 2

version: '2'
services:

  svc-using-svc-name:

    ...

    networks:
      - default
      - specific-network-name
    external_links:
      - exact-container-reference

    ...

networks:
  specific-network-name:
    external: true

edited Mar 02 '18 at 07:53

answered Mar 01 '18 at 17:58

helvete

2,455
13
33
37

2

Is ths `external` property false by default ? – Stephane Jan 28 '19 at 14:31
3

@Stephane: Yes, `docker-compose` assumes you are about to configure the project-specific network by default. By specifying external `docker-compose` won't try to create it, just to connect (and won't touch it in case of calling `down`) – helvete Jan 28 '19 at 15:17
this don't work for me: first file run correctly, but when run the second file docker responds "specific network name" declared as external but could not be found. [RESOLVED]:Docker-compose add directory name to named volumes and networks, so in second file i must add project prefix to network name – Domenico Monaco Mar 13 '19 at 09:13
@DomenicoMonaco: Then you are probably doing somethig wrong - check your docker versions, list docker networks, check correct compose file versions and you will see the network name is not prefixed w/ any directory name. May I also assume the downvote came from you? Why - because of misunderstanding on your side, perhaps? Have a nice day. [EDIT:] As a side note, this setup is our production model for several applications, nothing posted without testing... – helvete Mar 13 '19 at 10:07

score 7 · Answer 3 · answered Apr 09 '20 at 14:55

7

I also had this error:

Network "NETWORK_NAME" needs to be recreated - option "driver" has changed

   docker network ls
   docker network rm NETWORK_NAME
   docker-compose up

After removing the problem network and recreating it, things were fine.

answered Apr 09 '20 at 14:55

Rob Evans

2,822
1
9
15

Exactly what I was looking for! Thx! – Oo.oO Dec 04 '20 at 11:26
1

you can also use `docker network prune -f` to remove any networks with no containers attached. I sometimes do `docker rm -f $(docker ps -a)` to remove all containers. followed by `docker network prune -f` to remove the unused networks – Rob Evans Dec 04 '20 at 13:41

score 5 · Answer 4 · answered Jan 11 '23 at 20:25

Do not conflate external: true with internal: false. To allow separate compose projects to share a network connection AND have that network be automatically created, simply specify a matching network name in each compose file.

internal

when set to true allow to create an externally isolated network

By default, networks are NOT internal and thus already provide external connectivity.

external

If set to true, external specifies that this network’s lifecycle is maintained outside of that of the application.

The external property refers to how the network is managed and does NOT describe its connectivity or lack thereof.

external: false:

is the default and can be omitted
allows a network to be auto-created as needed
will not throw an error if the network was previously auto-created by another running stack

name

Absent a specified name attribute, auto-created networks use the key prefixed with the project name. Explicitly setting the name prevents project scoping and allows for shared use.

Example

project1

services:
  db:
    networks:
      service_tier:
        aliases:
          - remotedb

networks:
  service_tier:
    name: service_tier

project2

services:
  web:
    networks:
      service_tier:

networks:
  service_tier:
    name: service_tier

Result

$ docker network ls

NETWORK ID     NAME                  DRIVER    SCOPE
ca564d4e76e6   bridge                bridge    local
6b4d2866afa8   host                  host      local
15fa57f54e0b   project1_default      bridge    local
a5eb24fc02b0   project2_default      bridge    local
d4b0a10c1385   service_tier          bridge    local
614c5864df80   none                  null      local

In this configuration the web service in project2 can reach the db service in project1 by using the host alias remotedb on the shared service_tier network.

Without name in the networks sections you would see separate project1_service_tier and project2_service_tier instead and communication would not be possible.

score -11 · Answer 5 · answered May 07 '19 at 12:27

-11

Run the following command: docker network create ddev_default

answered May 07 '19 at 12:27

jcalabria

1