Securing your credentials and environment variables with AWS

Access management, secure storage, key rotation and managed encryption are as easy as ticking boxes since we migrated our hosting and workloads to AWS!

At Tigerspike, Security is a top priority alongside Improving People’s Lives Through Technology. And when we say “People”, we mean people who are benefiting from the digital solutions we bring as well as our people who put these solutions together.

One of the security best practices we enforce is using AWS Systems Manager Parameter Store to store any sensitive app configurations. We can then either use AWS SDK to retrieve these secure strings, pass them directly as environment variables for the app or compile them with AWS CLI on boot depending on the application. Let’s dig deeper into those three methods and see how easy it is to secure your app’s credentials.

First and foremost, check out how to create and organise your Parameter Store into hierarchies. Once you’re familiar with that, let’s assume you have two secure parameters one is /coolapp/dev/database/password and the other is /coolapp/prod/database/password. This hierarchy is just an example and of course you can organise whichever way makes sense to you. Sorting by app then environment then service makes it easier for me to find parameters in accounts that host multiple apps.

Now, let’s look at using AWS SDK to retrieve these secrets

AWS SDK is available for most, arguably all, the commonly used programming languages out there. Find the list here.
So, with the SDK, you can make calls from your app to get any secret from the Parameter Store whenever needed. For example, for Python:

import os
import boto3
ENV_NAME = os.environ['ENV']
client = boto3.client('ssm')
DB_PASS=client.get_parameters_by_path(Path='/coolapp/'+ENV_NAME+'/database/password',WithDecryption=True)['Parameters'][0]['Value]

This script above reads the environment name from the environment variables available for the runtime (so we can read for dev as well as prod dynamically) then calls Systems Manager to get the DB Password parameter by its path. Simple!
More on this here for Python, here for .Net, and here for JavaScript.

Next, let’s look at passing these secrets as environment variables directly

Most of the commonly used programming languages support the OS (or runtime)’s environment variables as configurations provider. Python or .Net Core for example. So, to utilise this feature properly, configure your AWS resources to feed environment variables directly from SSM. The environment variables configuration is available to many AWS services including ECS Tasks, Lambda Functions, CodeBuild Projects, and some more. And if it’s not available through the console, it’s almost certainly available through smarts and the use of CloudFormation intrinsic functions. For ECS Tasks for example, you can define the following which will make the secret available in the container as an environment variable:

Task:
  Type: AWS::ECS::TaskDefinition
  Properties:
    …
    ContainerDefinitions:
      …
      Environment:
        -Name: ASPNETCORE_ENVIRONMENT
        Value: !Ref pEnvName
      Secrets:
        -Name: DatabaseSettings__Password
        ValueFrom: !Sub arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/coolapp/${pEnvName}/database/password

How easy was that?!
Check here how to specify sensitive data for your ECS containers.
And check here how to use dynamic references in CloudFormation in general.

Now, what can you do if your application is written in a language that doesn’t support environment variables as a configuration provider or if the host you’re running your application on can’t take environment variables dynamically like the above?

Let’s look at using AWS CLI

With AWS CLI, you can write a custom script that runs first thing when your instance initialise. This script should simply get the required parameters then either write them to the app configurations file or export them into environment variables if the app supports. Here’s an example for a Bash script that we can use to get an entire secrets file from the Parameter Store and write it to a local file before starting the application:

aws ssm get-parameter -name /coolapp/dev/secret -with-decryption > response
cat response | jq ‘.Parameter.Value’ | jq ‘.|fromjson’ > secret.json

Using this concept, we were able to write a Powershell script for one of our recent builds that has to run on .Net Framework which rewrites the entire web.env.config XML file before running the infamous XML transformation with the wb.config. You get the gist. We didn’t have to store the secrets in the codebase anymore.
Here is more on AWS CLI and AWS Tools for Powershell.
And here is the Get-Parameter CLI command and Get-Parameters-By-Path if you want to compile values from different parameters.

What else to consider?

When working with Systems Manager Parameter Store, consider restricting your IAM policies to parameters by path, or part of the path. E.g. give access to arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/coolapp/dev/* only to your Dev instances and resources. This will guarantee keeping your production parameters secure from the your Dev and will also ensure your Dev apps will not have access to your production database, for example, by accident due to wrong use of the CLI or SDK or mistyping of a parameter path. More on this here.

That’s it folks!

It’s never been this easy and fun to hide your secrets. Pick and tweak for your application.

Edit: this User Guide has pretty good links to Learn More about Parameter Store, Getting Started, Walkthroughs, and Use with AWS KMS.