Setting up a project with a security-hardened CI/CD pipeline
A Beginner’s Guide
“Every journey begins with a single step.” - Lao Tzu
In our daily work, we often come across projects which have been started by developers that are looking to quickly code features and build software that can be deployed quickly. The focus early is on shipping something, and as a result there are many security and automation considerations which should be taken into account, but are not at the expense of going faster. These early decisions and considerations to prioritize speed make it more difficult and expensive to alter in the future (this is known as technical debt).
Many of the modern code repository systems have built in building tools that are like a gift from the DevOps goods:
- CI/CD pipelines, directly included in their repository provider (GitHub Actions, BitBucket Pipelines, GitLab CI/CD pipelines).
- Containers and container orchestration systems.
Source-integrated CI/CD pipelines and container orchestration give development teams the building blocks making it possible from day one to map out a scalable production system that meets many of the requirements for future compliance needs. The early investment in automating the production build pipeline, can improve the speed of deployments (the legendary 10 deployments per day goal), and the confidence in the correctness of what is deployed. The tools support the team to ship more faster from day one. Allowing the team to grow and the pipeline automation to support the growth from the start.
In this article, we show an example of how to set up a software project to use automated CI/CD pipeline to build and deploy to product in the “right way”, to get started. Setting up a build process should take less than a day. Here is how to get started and automate your build pipeline.
Considerations and Stack Decisions
To demonstrate the ease with which this can be accomplished, we are going to start a simple React application project.
We are going to add recommended components, and set up a baseline for a CI/CD pipeline, which continuously checks both code and the code defining infrastructure.
We will be focusing on the setup of the CI/CD pipeline. The fixing of the issues with regards to code and configuration scans will be subject of a subsequent article.
Here are our choices of tech stack:
- GitHub, using Actions/Workflows, as our choice of CI/CD and repository.
- As a code coverage tool, we are going to use the `react-scripts test –coverage` included in the create-react-app template
- As a linting tool, we are going to use Prettier.
- We will perform basic code scanning using Semgrep
- The deployment strategy will be done using Docker containers. To simulate an environment, we will use Docker Compose.
- GitHub Secrets will be used to store our secrets
- For infrastructure/configuration/Docker image scanning with CoGuard.
- We will create a Redis instance for shared state
- The React App will be sitting behind NGINX, which also serves as a load balancing mechanism
- The monitoring will be performed with the ELK stack.
- For authentication, we are assuming that there will be an OICD provider given which we can configure. The react application will use passport.js for the authentication logic and framework. The details will not be covered in this article.
- Since we are just looking at a plain front-end for now, we are going to omit any database related work.
Step 1+2: Set up code repository, and start with a code coverage workflow
Create a folder on your machine, and execute the following commands inside of it
The link at the second command is the supplementary git repository we set up for this blog article.
After installing `create-react-app`, simply run
To install the basics of a react application.
After this is done, we should set ourselves up with a good coverage and code fail threshold.
In the file `package.lock`, add the following key-value pair.
Now, in order to have test coverage from the start being part of the build and fail when not, we are going to create our github action to test that on each pull request.
For that, create the folders .github/workflows, and inside we will have a file called coverage.yml.
In order to get the basic coverage report for your project, coverage.yml needs the following lines.
The test is initially failing, since the coverage is below the threshold. As mentioned before, we will deal with the fixing in a future article.
Step 3: Linting
In order to add prettier to our project, we simply run
and create a new file .github/workflows/prettier.yaml with the following content:
This job fails from the start as well. As you can see, we also added ESLint here, since it also contains checks for potential bugs.
Step 4: Add SemGrep as a code scanning tool
In order to use SemGrep on our code-base, we will simply install the action from the marketplace. Hence, we would add a file called .github/workflows/semgrep.yml with the following content.
Step 5: Create a basic Docker container for the react app, and add it to a docker-compose
The basic Docker container should already follow some good principles. We are putting the Dockerfile at the root of the repository, and it contains the following lines.
And the simple docker-compose.yml will look like the following:
Steps 6+7: Infrastructure Scanning initialization and Secret Storage
We are using CoGuard to scan our Dockerfiles and images. For that, we are going to set up two secrets, namely the CoGuard username and Password. Then we are going to add to the workflow the upload of that specific Dockerfile to start.
After creating an account, we store the username and password inside GitHub Secrets with the identifiers
Now, we are creating an action to upload the data to CoGuard. This is done using the following action script:
Steps 8+9: Adding Redis and NGINX
We will add Redis and NGINX as containers. However, we will also ensure that their respective configuration files are tracked in our repository.
We will store them inside subfolders in docker_images/nginx and docker_images/redis.
Let us start with nginx. We will right away assume that there will be TLS traffic enabled, and hence, we will need our first certificate pair.
We will generate it inside docker_images/nginx/conf.d.
The command to generate one is the following:
The Dockerfile for nginx looks like the following:
We will do the same with Redis.
The Dockerfile for Redis is like the following:
The configuration redis.conf is stored in the subdirectory, and copied (for now) from the redis example configuration as found here: https://redis.io/docs/management/config-file/
The docker-compose entry for redis is looking like the following:
Our goal will also be to have the configurations scanned by CoGuard. The additional lines in .github/workflows/coguard.yml are the following:
Step 10: Monitoring with the ELK Stack
We are not going to put a full-on description here on how to include the ELK stack into the docker-compose. We will, for now, follow the instructions as given here and talk about the alterations needed.
The alterations to get this working in the docker-compose file is to add the `network` key to every service, and provide proper IP addresses. As with the services before, we will discuss in a subsequent article how we are going to secure the configurations of each element.
As per always, we would need to be able to be in control of the individual configurations, and hence, we create our own Dockerfiles with versioned configurations.
And, the elasticsearch.yml and kibana.yml are the default ones coming from the container for now. In order to include them into the scanning, we are adding the following lines to coguard.yml:
This article demonstrates how to start building a react application, get it committed to GitHub and set up linting and code scanning. Then we set up and use GitHub Actions, Docker, Docker Compose, GitHub Secrets and CoGuard to build and secure the initial CI/CD pipeline. This initial investment and steps has laid the groundwork for a faster deployment pipeline. It has added key tools and items to help the team build faster and reduce the risk of accidentally introducing bugs.
All of the configuration files, starting from the Infrastructure as Code (IaC) layer down to the individual configurations of infrastructure and containers have been versioned and added to source repository. This is the foundation for the development teams to grow and ship code securely.
The full repository for this blog article can be found here. This repository includes all fixes and status completed at the end of the article.
In subsequent articles, we are going to tackle resolving the issues flagged by the different scanners/coverage checkers. This will include configuring the build pipeline to terminate the build workflow with breaking issues allowing the pipeline to create tickets in the issue management system allowing a developer to build on top of it. We will also publish an article outlining the reasoning for each toolset.