AWS: Correct permission to start an EC2 instance with encrypted EBS

AWS has a very flexible permission system using IAM policies. But sometime it is also complicated to get the access control right. Yesterday, I worked on a IAM policy to allow an instance start / stop another instance.

It should be straightforward, and this is my initial policy.

{
    "Effect": "Allow",
    "Action": [
        "ec2:StopInstances",
        "ec2:StartInstances",
        "ec2:RebootInstances",
        "ec2:Describe*"
    ],
    "Resource": "*"
}

It worked well. But I quickly found out that it didn’t worked for all instances. It didn’t provide any information, the instance quickly entered pending state then stopped immediately with message Server.InternalError: Internal error on launch. If I started the instance from the console, the instance started up just fine.

After some investigation, the above policy would not work with the instance has an encrypted EBS. So I go ahead and updated the policy to allow KMS key usage, as described here.

{
    "Effect": "Allow",
    "Action": [
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:Encrypt",
        "kms:DescribeKey",
        "kms:Decrypt"
    ],
    "Resource": "{{KEY_ARN}}"
}

But it still didn’t work. If i change the permission action to kms:* then it worked. After a lot of trials, errors and documents, I found out that you would have to allow kms:CreateGrant permission as well. So basically, when you start the instance you are also delegating the encryption permission to the instance so that it can encrypt / decrypt the EBS, hence you will need the CreateGrant permission.

My final policy looks as follow, and it works for all the cases so far 😀

{
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:StopInstances",
                "ec2:StartInstances",
                "ec2:RebootInstances",
                "ec2:Describe*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:RetireGrant",
                "kms:CreateGrant",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:Encrypt",
                "kms:DescribeKey",
                "kms:Decrypt"
            ],
            "Resource": "{{KEY_ARN}}"
        }
    ]
}

I hope this post will save someone time, because it takes me 2 hours to figure out the problem!