OC Continuous Integration is an organisation problem
After 10 years in DevSecOps, there is an assumption that that every project is unique and needs to deploy their own Continuous Integration (CI) instance and write their own Continuous Integration/Continuous Deployment (CI/CD) pipelines.
However within a CI Pipeline you should be producing a build artefact which is supplied into a CD pipeline. Many CD Pipelines can be triggered by various means. This allows you to manage CI and CD pipelines separately.
While the mechanism and configuration of a deployed product can vary greatly between software products the CI pipelines face the same constraints which mean each project implementation is highly limited in the process it must implement. This means each organisation can solve the problem once and reuse the solution between all teams.
There are only so many build systems
Modern software languages have build automation systems and dependency management systems, these aim to automate the mundane tasks in building, testing and releasing a software project. The result of a build automation system is a 'build artefact', this is a container/wrapper for the software which can be executed or referenced by other software projects.
The key advantage of using a common system over a bespoke solution is it becomes easier to hire staff with those skills, you don't have to support the build automation system and there are external training and support options. As a result every software programming language will focus on a few build automation solutions.
Build systems are very similar in how they work
From Ivy to Setuptools each build automation system expects a configuration file (ivy.xml, package.json, rubygems, setup.py, etc..) with the details required to build, test & release a software project. This means any Continuous Integration pipeline for Language X and Build Management Y should call the same sequence of commands.
The fact we have a defined set of build steps, means there is a limited number of permutations for building a software project using a specific build automation system.
At this point we could argue one person could write a CI Pipeline for each language/build/CI system and we should all reuse it, however...
Organisation variance
Each organisation will have its own processes, requirements and constraints. This is because each organisation can structure itself in various ways and a lot of those roles will define their requirements.
This re-expands our problem space, so trying to build one solution for everyone becomes impossible. However within an organisation there are only so many requirements, processes they will expect you to follow.
Trying to build a CI pipeline for X language/frameworks for multiple teams means you will need to understand the organisation processes and requirements for each team.
Most corporate processes are created and never reviewed. The act of creating an organisation solution will mean you review those processes and can lead to lots of quick wins.
For example I have automated the generation of release design docs dozens of times. One job had a really convoluted word doc and most teams dedicated an engineer to the task of writing it for each release.
I sat 3 service managers down, discovered they wanted to know what the version was, what issues were in the version and what tests were run against a version.
We paid for Atlassian Jira and the X-Ray plugin, when were performed a release we marked tickets associated with it and test engineers linked xray against the jira release. A 10 minute lesson on Atlassian Jira and the document was irrelevant.
But my team is...
Lastly each team can work differently but no matter how unique and special a team is they will need the ability to perform a release and the ability to verify a current build.
That doesn't stop the team adding additional CI pipelines, but it does mean the team have 2 less pipelines to write.
Ensuring everyone verifies code using the same rules and and tools produces consistent measurements between teams. This allows you to understand the state of software within an organisation.
Lastly some will argue they don't need a heavyweight process. If you solve CI at an organisation level and provide it as a service, it reduces the effort a team has to spend. It doesn't matter that the process performs a million tasks, the team didn't have to write it or maintain it. They can take what they need from the results
Add comment