Today, I will show you one way in managing your multiaccount organization in AWS. Say you have setup your new AWS Control Tower and started to provision accounts into your new Organization. Great! But how do we manage it and take control of our underlying accounts? We have a few tools in our toolbox to help us with this, for example:
- AWS Config - Audit and save all configuration changes in the environment and send alerts for further actions.
- AWS Organizations - Or rather, the Organization feature Service Control Policies(SCP). Basically this acts as a filter with policy denies that will explicitly deny certain specified actions in underlying accounts and organizational units.
- AWS CloudFormation - Or rather, CloudFormation StackSets that will integrate with the service Organizations to deploy resources into the underlying accounts. This is going to be the topic for the rest of this post.
I will demonstrate a really simple but versitile use case that can be easily expanded into other areas. Our fictional organization has a lot of developers and our applications are mostly placed on EC2 instances. We want to provide lab accounts to all the developers but we want to make sure that we can have some limitations in order to minimize cost. In our case, we have agreed that all EC2 and RDS instances will be stopped every day at 00:00.
We could do this in multiple ways. For instance, we could spin up infrastructure in a totally seperate management account to then speak to the AWS APIs against the other accounts to stop instances, etc. But remember, one of the key features of a multiaccount strategy is to create security boundries between AWS accounts. Hence, we cannot actually access other accounts without those accounts explicitly grant our account permissions. We would have to ask all the developers to grant us permission which is of course not a very secure and scalable solution.
In comes CloudFormation StackSets. If you’ve ever deployed a CloudFormation stack before, you’ve basically done almost all the steps required to deploy a StackSet. The biggest difference is that you can also configure Organizations integration and some rate limiting.
Before going through the deployment process, we’ll look at the source code. I will not try to discuss the template in detail but just go through in general the components and what they do.
This will contain 3 files:
- StackSet.yml - This is the CloudFormation stackset to be deployed.
- lambda_function.zip - This is just a zipped archive of the lambda_handler.py python file that will exist in a S3 bucket.
- lambda_handler.py - This is the actual Lambda code that will run
Basically, the entire solution depends on 4 components. The S3 bucket object(lambda_function.zip), the lambda function, an IAM role to allow all the actions to be formed and lastly the CloudWatch Events cron scheduler for actually scheduling the lambda function. And everything is created in each of the AWS account but managed centrally from the master account using CloudFormation StackSets.
One thing to remember is to place both S3 bucket and CloudFormation StackSet in the same region as you want the solution to be deployed in, otherwise you’ll get an error. Most suiteable is to place it in the region where Control Tower has been deployed.
Let’s have a look at the StackSet template.
We’ll start by taking in 2 parameters, S3Bucket and S3Key, where we can specify where Lambda will pull down the source code for the Python function to perform our task. So this is dependent on that the .zip file is uploaded to a S3 bucket. But you don’t have to make the object public to the internet for all Organization accounts to access the file. You can use the PrincipalOrgID to allow all accounts in your organization access to the object.
Then we’ll create the OrgEC2MgmtFunction Lambda function. Nothing special, it will use the parameters specified above to retreive the source code. It will also use Python 3.8 runtime and an IAM role that we’ll create further down.
An IAM Role OrgEC2MgmtRole is created where we allow Lambda and Events to assume the role. This is because CloudWatch events will be used to cron trigger the Lambda job in a later stage, which require the role to be assumed. We’ll allow CloudWatch logs permissions. Ideally, we would have created a log group beforehand in the template and then explicitly specified the ARN resource to that particular log group. Well, improvement for another time. Then we’ll allow all the actions required to perform all the actions we want. For instance, describe and stop all EC2 and RDS instances.
Lastly, we have the actual event that will schedule the Lambda function. Pretty much self-explanatory, it will use cron schedule “cron(0 0 * * ? *)” and use the role created earlier to schedule the Lambda function.
This is a really basic Python script that will use the IAM role provisioned using CloudFormation and list all regions and then going through them all to list all EC2 and RDS instances/clusters. Then it will go through all of them to stop them. It will also print out logs so it will be collected in CloudWatch Logs.
Time to deploy! Log in to the master account and go to CloudFormation. Remember to select the correct region.
Open the left menu and select StackSets, then select Create StackSet. Now you can upload the StackSet.yml file by selecting Upload a template file. Enter a Stack name and then the S3 object parameters. Click next until you get to the step Set deployment options. Here, you’ll select what kind of deployment you would like. You can either select entire organization or specific OUs. In this case, we want to use specific OUs so we’ll select that. Then we get annoying text boxes that we’ll have to manually fill in the OUs we want and there’s a limit of 10.
Easiest way to find the OU id is to open an new tab and go to Organizations -> Organize Accounts and then select the wanted OU. You’ll see the ID of the OU in the right sidebar. It looks something like this ou-xxxx-xxxxxxx. Copy and paste this into the StackSet OU fields and Add more OU fields as needed. The rest of the settings, I usually leave on.
Then we’ll specify region. In this case, select only the region that you want the solution in as we only want one solution per account.
In the bottom, we can configure deployment options if we would like the deployment to happen faster by deploying to a larger number of accounts simultaneously.
Next we’ll review all the settings, acknowledge that CloudFormation is allowed to create IAM entities and then press Submit.
We’ll now see that the solution is deployed into every account in the select Organizational unit and it should perform as intended. Obviously, this needs to be tested on test OUs before deploying into environments that affect people. Try it on yourself first. :-)
You should use Organizations Service Control Policies(SCP) to limit users to be able to interact with the deployed resources in accounts. You can do this by limiting by resource name or by utilizing a tagging strategy.