Steps toward automation #
In this blog post, I’ll be discussing my approach to automating tasks that
software engineers have to perform for their job. Before diving into the core
concept, I’d like to define and narrow down the idea of automation within
Automation, in this context, refers to code created by programmers to simplify
tasks. The users benefiting from this automation are the computer programmers
themselves. The platforms on which programmers automate tasks include
development machines, CI/CD systems, cron jobs, and more. Typically, there is a
trigger for the automated task, such as an updated file, a new version of a
container, or the presence of a binary file. The job may involve security
scanning, updating downstream dependencies, or sending Slack messages.
Automation complexity arises when scripts have interdependencies, often
represented as a pipeline in a CI/CD system like GitHub Actions, GoCD, or
Concourse. An example pipeline can be a new git commit that happened in the code
base, tests need to run against the code, and when they pass, the code gets
deployed into an environment, which has performance and end-to-end tests to make
sure there was no regression. If anything fails in the pipeline, you can rerun
any part of the pipeline just in case.
As a software engineer, I’ve established two ground rules around automation.
First, “Automate yourself out of a job,” a mindset more than something precisely
actionable. If you find yourself repeatedly performing a task that doesn’t
require critical thinking but still consumes time, try to figure out how to
automate it. For example, if you’re always responsible for deployments and it
takes up a significant portion of your time, automate the process to free up
My second rule for automation is, “Only to automate things you can do manually.”
Many engineers, myself included, have fallen into the trap of automating a task
before fully understanding the problem, leading to spectacular failures when
there’s no manual fallback. To avoid this, ensure you know the commands and
requirements for manually executing the automated task in emergencies.
Applying these two rules will help you identify the repetitive tasks you’re
performing and automate them with confidence. Enabling you to share your
understanding with team members and provide context.
I once worked on an application where our platform was a CI/CD system that
allowed people to install and update a platform. We provided bundled scripts,
which meant most users never saw the commands contained within the scripts. We
embedded these scripts and commands in our documentation to maintain
transparency, allowing users to run them from their development environment.
We wanted to prevent users from encountering issues when copying and pasting
commands from the documentation, such as syntax errors or incorrect arguments.
Applying the two rules ensured the commands worked manually and within our CI/CD
system. First, we manually tested the commands before adding them to the
documentation. Second, we used the bundled scripts in our CI/CD system to
confirm they also worked there. This approach led to documentation with its own
CI/CD, ensuring everything was valid and functional. It was an advantageous
With any rules, there are always going to be edge cases. For example, there may
be security constraints with running specific code. The condition requires that
code be executed within a particular environment because the credentials and
network access have been isolated. You’ll never be able to run this locally from
your development machine.
Also, with manual and automated intervention, there will be times when you want
to do something manually constantly. It could be for the satisfaction,
validation, and verification that it Just Works™ correctly. It could also be
that you are the only one that runs this. Where the line exists between
automation and manual execution is a personal/team preference; just make sure to
check yourself that your automation is not over-engineering a straightforward
In conclusion, automation in software engineering can be a powerful tool to save
time, reduce human error, and streamline processes. By adhering to the two
ground rules of automation and considering edge cases and personal/team
preferences, you can create a robust automation system that benefits you and
your team. Remember, automation aims to make life easier for you and your fellow