Summary

The script added allows us to structure our .env files a bit better by splitting secret variables out of the main .env, generating the final .env by merging the non-secret and secret variables and do a final validation that the generated .env matches a template for what it should look like (ie. it has all the same variables).

Hopefully this will lead to us being able to

1. More easily code review .env changes as:
   1. You will be able to see the diff on changes to the non-secret .env's (currently, we encrypt the whole .env, which means that you blankly stare at a code review not knowing what changed and whether something significant changed or not and not being able to review the change)
   2. Even if the secret .env's are encrypted, the variable name is in the filename, so you can discern the incremental change (ie. Gabor: "Oh, I see you changed the DB_PASSWORD.env file, I guess that means we are changing our database password?" Hans: "That is correct!"), without the internal secret being exposed.
2. Allow us to have more structure around changes made to the .env's. For instance, in the process of building this tool, I noticed that some .env's (most notably the homestead one) had a ton of missing variables and even a few that are no longer useful. This tool will validate that the generated .env follows the specification, so that our different .env's are always clean and conformant.

Details

The way this works is that you execute the following command within your project (same applies in the CI pipeline) something like this (since it is a vendor package) from the root of your project:

./vendor/bin/build-and-validate-dotenv-configuration -e homestead -l ./config/dotenv -o ./

What it does next is:

• Generate the .env by assembling the non-secret .env along with appending the secret variables at the end of the generated .env and load it into a EnvironmentConfiguration object, which parses the .env - extracting the key / value
• Load the template.env - extracting the key / value
• Pass these two to the validator, which compares the two and makes sure they both contain the same variable names - this enforces the structure of our .env file
• Finally, it outputs the generated .env to the output location from your arguments

If anything wrong happens, you get a big fat error message, but, more importantly, if the generated .env does not match the structure of the template.env, you get an informative error message telling you which variable names are offending. :)

Note: It is important to note that you are still required to use ansible-vault decrypt to decrypt your secrets before you run the build-and-validate-dotenv-configuration script. I have already incorporated this into our CI pipeline for Acadia, but the core difference is that, instead of decrypting a single file, you are now decrypting all files within a directory by using the wildcard, *.

ie. ansible-vault decrypt path/to/secrets/* and then you will be prompted for the password - as per usual

This merge request also includes test .env data that you can look at to see what the new structure looks like. It is basically:

./template.env (this is the master template, which the generated .env gets compared to)
./development.env (this is the non-secret .env named after the environment)
./development-secrets/secret1.env ("secret1" is the variable name, while the contents are its value)
./development-secrets/secret2.env ("secret2" is the variable name, while the contents are its value)
./staging.env
./staging-secrets/secret1.env
./staging-secrets/secret2.env
etc...

Future development

• Currently the validator only validates that we have matching variable names, but nothing further. It would be nice if we could also validate that the .env is valid and does not contain any syntax errors.
• Maybe we should consider incorporating the Ansible decryption as part of this tool. It wouldn't be very hard.

Thoughts

• Having a template.env & non-secret .env at first might seem like 'just another thing you have to do'. It does mean that, on any change to the .env's, you also have to update the template.env. I struggled a bit thinking of a better way, but every option I considered would require us having to programmatically generate a .env file and the phpdotenv package that Laravel uses is merely a parser and so my thinking is that this is beyond the scope and perhaps a future consideration - if it really is worth it at all. In the meantime, it really isn't that much more painful to do one extra thing (updating the template.env as well) and, at the very least, the CI pipeline will let you know if you missed that step! And the benefit of having a clean, structured and reliable (emphasis!!!) set of .env's plus the ability for us to actually be able to code review them far outweighs the additional work involved... IMO!