CNI Plugin creates publicly readable log files, failing Inspector CIS rules.

[linxcat]

What happened:
In an effort to harden our EKS cluster to CIS standards, we have employed Inspector in discovering weaknesses in our custom AMI implementation. Rule 4.2.4 in particular is proving very troublesome, since the failure is part of the base AL2 EKS AMI itself. The CNI plugin creates files in /var/log/aws-routed-eni/ which all have globally readable permission, failing AWS's own tests. I hope there's a way to configure it not to do this, but I've been searching for nearly a whole day and have found nothing at all documenting it.

Environment:

• Kubernetes version: v1.15.12-eks-31566f
• CNI Version: v1.5.7
• OS: AL2
• Kernel: 4.14.209-160.335.amzn2.x86_64

[jayanthvn]

Hi @linxcat

There is an option to disable this logging (AWS_VPC_K8S_PLUGIN_LOG_FILE) and redirect it stderr. But yes as you said currently it is global readable -

ls -ltr | grep ipamd
-rw-r--r-- 1 root root 3271068 Jan 14 17:40 ipamd.log

Are you looking for an option to make it readable only for say root user?

Thanks!

[linxcat]

That would be the most appropriate thing, yes. That being said, I'd also like to know the location of that option (file, env, etc) for alteration if necessary.

[jayanthvn]

Currently I dont see an option which makes the file readable only for a user other than manually doing a chmod on the file. Let me dig deep and see if something can be done here.

[jayanthvn]

Thanks @nithu0115 for sharing this. Currently zap logger doesnt support setting file permission - uber-go/zap#860. One alternate way we think is to create the directory and file ahead of time.

[linxcat]

Buy this do you mean on image build (with packer for instance)? When is this folder created automatically, during EKS bootstrap?

[jayanthvn]

This can be done at bootstrap, but the recommended way is to add a pre-start script in the init container (https://github.com/aws/amazon-vpc-cni-k8s/blob/master/scripts/init.sh) to just touch this file and chmod to change the permissions. The folder is created by aws-node ds (https://github.com/aws/amazon-vpc-cni-k8s/blob/master/config/master/aws-k8s-cni.yaml#L186)

[jayanthvn]

Hi @linxcat

Did you get a chance to try this recommendation of file touch with init container? Please let me know if you need any steps to add that.

[linxcat]

I've just been able to come back to this issue after other concerns took priority. If I understand this correctly, this means adding an init container to the aws-node daemonset and which would affect all nodes launched post-change. I can see the init container mentioned in this daemonset, but I don't have any knowledge on what adding a script to it (rather than an extra initContainer) entails.

[jayanthvn]

Hi,

I did a quick POC and modified the existing init container and that seems to work -

[root@ip-192-168-72-127 aws-routed-eni]# ls -ltr
total 20896
-r-------- 1 root root 21396088 Mar  5 07:50 ipamd.log

Changes - In https://github.com/aws/amazon-vpc-cni-k8s/blob/master/scripts/init.sh, I added the below lines -

touch /host/var/log/aws-routed-eni/ipamd.log
chmod 400 /host/var/log/aws-routed-eni/ipamd.log

And in the DS had to mount the directory in the initContainers section -

 - mountPath: /host/var/log/aws-routed-eni
            name: log-dir

Yes another initContainer needs to be added to achieve this functionality and not just the nodes launched post change, it will restart aws-node on the existing nodes in the cluster and set the file permission appropriately.

[linxcat]

This solution you're describing involves branching out and maintaining this new version of the script, correct?

[linxcat]

Also, your solutions have focussed on specifically the ipamd.log file within /var/log/aws-routed-eni/ as I mentioned in my original comment, but there are dozens of files in there besides the one you are mentioning, including the folder itself, which have this readability problem. There is one file for every single eni on the node. They're also all uniquely named (after the eni ID) so pre-touching them is impossible.

[jayanthvn]

Yes, this would need special management of the script and you can chmod on the main directory. CNI manages only ipamd.log and plugin.log. Unfortunately until we get support from zaplogger infra [https://github.com/uber-go/zap/issues/860] we cannot achieve this by default.

[jayanthvn]

We will keep track of the zaplogger infra bug and update the permissions once we have the support. Please feel free to reach out.

[tehmoon]

I'm having the same issue with aws inspector. I did fix the permissions by creating the directory and the files prior to being used which seemed to have done the trick. The issue that I'm having is that I'm still not compliant. I have done everything I could think of and it still complains. I understand this is not an error related to this project, but it's the only mention I have found anywhere that is close to my issue. Are you passing 4.2.4 now @linxcat ?