Introduce support for external EFS volumes

[ndeloof]

What I did
Introduce support for EFS volumes set as extrernal volumes:

services:
  foo:
    volumes:
      - "data:/tmp"

volumes:
  data:
    external: true
    name: "fs-1234abcd"

Related issue
https://github.com/docker/ecs-plugin/issues/254
https://github.com/docker/ecs-plugin/issues/220

(not mandatory) A picture of a cute animal, if possible in relation with what you did

[gtardif]

How does name: "fs-1234abcd" link to the EFS? (or what are the requirement to create a valid volume we can point at?)
Just to understand a bit the magic behind this.
I guess there is no credential issues because your AWS profile must have access to the EFS volume, and security rules are defined at deploy time to resolve access from the containers

[gtardif]

hmmm... what's this ? :)

[ndeloof]

temporary workaround waiting for awslabs/goformation#300 to be fixed

[gtardif]

why requiring a string[] as parameter if we only use the first element ? (opened question, is it just because the AWS API returns a string[] in WithVolumeSecurityGroups? )

[ndeloof]

yes indeed, I'd like to keep sdk as a thin wrapper on aws-go-sdk and not introduce too much logic there

[ndeloof]

How does name: "fs-1234abcd" link to the EFS? (or what are the requirement to create a valid volume we can point at?) Just to understand a bit the magic behind this.

this is the filesystem ID (unique). As an external volume, setting name allows to distinguish between compose model internal references (service.volumes -> volumes) and external identifier that might not be a valid compose ID.
see https://github.com/compose-spec/compose-spec/blob/master/spec.md#name-1

I guess there is no credential issues because your AWS profile must have access to the EFS volume, and security rules are defined at deploy time to resolve access from the containers

Nope, nothing helped resolve access from the container, that's why I have to register a dedicated ingress rule to allow NFS traffic between (at least one) service's securityGroup and EFS volume (a distinct mount target per A.Z)

[gtardif]

To be demoed on Thursday ? 🎉

[christophluehr]

Quick note, volumes can have dashes in their compose file names, with ECS/Cloudformation this leads to errors (if someone else runs into this):

    volumes:
      - data-vol:/data

volumes:
  data-vol:
    external: true
    name: "fs-xxxxxx"

ValidationError: Template format error: Resource name frontendNFSMountdata-volume is non alphanumeric.
status code: 400, request id: fb3c1-xxxxxxxx