Problem
Suppose Ahmed is working for a company whose success is dependent upon offering new and exciting products to its online customers faster than its competitor. Now Ahmed is a developer who writes code for new products, new features, bug fixes and security updates in the products. Unfortunately, Ahmed needs to wait weeks for his work placed in the production. And this delay increases the pressure of staying competitive because some competitors are able to deploy new products and new features much faster. In addition, this delay makes it very challenging for developers to manage both the code that is pending to be pushed for production and the development for the next feature, he is responsible for producing.
And when Ahmed's code is deployed in the production environment, occasionally unseen errors and problems occur. It mostly happens because Ahmed was just focusing on writing code based on the development environment and it is not identical to the production environment.
Now the Operations team comes into the picture, Zunair joined the operations team and he is responsible for 99.9% uptime of the production servers. And as the number of products developed by the organization is continuously increasing, it is very difficult for Zunair to maintain all the servers. Because the tools which he is using to administer all these servers are not so effective to administer all the servers. This creates the challenge for Zunair to migrate the new code into the production environment. And once a new code is deployed, it is his responsibility to monitor all the problems and errors occurred by this new change into the production. And Zunair has to do all Smoke test on each build into the production.
Sometimes, it also happens in the scenarios, when the teams have to manage four different environments like:
- Dev
- QA
- UAT
- Production
The Quality Assurance Engineer has to do all kinds of Smoke tests in the application in different environments. And it takes a lot of time and efforts of the entire team. Sometimes, there are a lot of problems in the shipments and QA has to identify all kind of problems by running those shipments in environments. Suppose there is something wrong in QA environment by running that shipment, then QA reports the issue to the Development team and then developers have to solve that issue with new efforts. And when developers are sure that now, things are working fine, then it will be run on QA environment and after passing the Smoke test on QA environment. Quality Engineer runs the shipment on UAT (User Acceptance Testing) and if things are working fine, then it goes to production, otherwise the issues are reported back to the development team.
This is how it takes a lot of time and effort of the team.
Time Consumption
We can conclude from the above example how much time will be consumed to move things into production. Tracking bugs are always very important in software development. Earlier, we tracked the bugs in the product, it consumed less time. The bugs which we track in the code during development is the best condition for saving time.
Now suppose buggy code goes to the QA environment and QA or Test engineers find out bugs in the code and report back to the developers. In this scenario, it consumes more time than in the first scenario.
Now suppose the bug in the code is not found out by the Test Engineers and the bug goes into production. And customers report the error and strange behavior of the application. And then, the client escalates back to the product team. In this scenario, it will consume a lot of time and effort of the team.
What is DevOps?
So here, DevOps comes into the picture to resolve all of the above problems. DevOps itself is not the name of any software tools. It is basically the name of culture. It is the name of a mindset to do the work. There are many definitions of DevOps on the internet. But I would like to quote only 2 clear definitions.
DevOps is the union of people, process and product to enable continuous delivery of values to our end user.
(Donavn Brown)
DevOps integrates developers and operations teams in order to improve collaboration and productivity by automating infrastructure, automating workflows and continuous measuring application performance.
If you read these two definitions one after the other, then things will automatically become clear to you. DevOps helps both developers and operations team to make the product mature. We always look for the solutions and strategies in how to deliver our product very quickly to customers.
And to make things faster, engineers started looking into Automation because only Automation is the way which can make things automated and it can save a lot of human effort and time.
Traditional vs Modern Methodologies
You might have noticed that in our industry, many organizations are moving towards Agile way of working. This means that the waterfall approach that we have embraced in the past decades is now fading. We are in the major paradigm shift; and we are moving in an engineering way of thinking about software delivery towards a manufacturing approach.
Traditional vs Modern Methodologies in Software Engineering
| Traditional | Modern |
| Waterfall Approach | Agile Approach |
| Different teams or organizational units for requirements, development, test and operations | Multidisciplinary teams where all disciplines work together on a small piece to deliver |
| Clear separation between Business and IT (demand/supply) | Business, Development and Operations in 1 team |
| Release software 2 or 3 times a year | Release multiple times a day |
| Budget/cost driven | Value stream driven |
So first of all, let’s discuss these difference before moving forward. In the Software Industry, the traditional way is we collect all the requirements at a time and then we follow the complete SDLC. And Agile is the way in which iterations and incremental approaches come into play. That’s why in Agile, requirement changes frequently and changes are always welcome. In Agile, the highest priority is to satisfy the customer.
In Traditional approach, there are different teams for requirement engineers, development teams, testing teams. And in modern methodology of Software development, Customer is directly linked with the developer to decrease the communication gap and to make the product according to the real and actual requirements of the customer.
In Traditional approach, there is a clear separation between Businesses team who is gathering requirements and the IT teams, whose responsibility is to supply the product. And in the modern approach, all these things are linked together at one place.
In the classical and older traditional approach, only 2 or 3 software releases were successfully released but in the modern approach, we can release features multiple times a day.
So it is important for all of us that we need to follow Agile methodologies and try to think out of the box to make things faster.
Agile and DevOps
Most of the time, people are confused and even they think that DevOps is a new thing. And sometimes, they are even confused about the difference between Agile and DevOps.
Actually, as I’ve already mentioned, DevOps is not a new thing. If we read the Agile Manifesto, we can get an idea behind Agile and DevOps.
“Our highest priority is to satisfy the customers through early and continuous delivery (CD) of valuable software.” (Agile Manifesto)
So it is concluded from the manifesto that when the Agile movement is started; they already explicitly stated the continuous delivery of valuable software. And continuous delivery is all about creating repeatable, reliable processes to deliver valuable software to our customers very quickly. And we have to automate nearly everything in order to achieve Continuous Delivery.
But still Agile and DevOps are two different things. Agile basically fills the gap between Business/Requirement Engineers and Developers.
Image: Guru99
And DevOps fills the gap between Software Developer and IT Operations team.
Image: Guru99
DevOps PlayBook
In Agile, there are predefined methodologies like Scrum and XP which are used to implement the Agile in the organization. Similarly for DevOps, there are not very well established, but most common methodologies that are mentioned below.
People / Process / Time
In this methodology, we identify the group of people who are responsible for the job functions, then we define the processes implementation by those people and then we specify the tools. It might seem very obvious to you but sometimes, Engineers and tech managers do the reverse steps and buy the tool first.
Continuous Deployment
It is just the practice of coding, testing and releasing software frequently in the form of very small batches to make the quality much better.
Continuous Integration is the practice of automatic building, unit testing the entire application and ideally, it happens on every source code check in. Continuous Delivery is the practice of deploying every change into production and performing automatic and integration testing on the code. Continuous Deployment is the practice of automatic deployment into production.
Most of the time, all the above three terms are used for same concept. There is not so much of a difference among them. And we if dive a little bit deep into Continuous Delivery, you’ll see the bigger picture here.
With the help of source control, after we checkin the code, the code build in the pipeline and all the associated unit tests run with that build. If all the tests are passes and code validation is ok, then it makes the package of the complete build and then it forms into artifacts. I’ll explain the artifact later.
Best Practices
When you’re making the build, there are some important points which we need to care.
- Build should pass in less than 5 min
The longer the build takes, the more people naturally wait until they have large batches of changes which increase your work in progess.
- Always commit small changes
- Don’t leave the build broken
If the test cases are failed or code is not compilable, then the build will be broken and don’t leave the build broken. If the build is broken, you block delivery. Stop all the meetings and stop all the team tasks until the build is fixed.
- Don’t allow strange tests
You might have experienced that someone writes the code which sometimes works and sometimes doesn’t work. So there is a problem in the code. Don’t write such kind of tests.
Once you have planned out your CD pipeline, always track changes. And ask two questions to yourself.
- Are you able to track changes in the whole system? (Cycle)
- How fast can you move your changes from development to production? (Cycle Time)
So we should have the matrices, how long does it take to flow the code from dev to production. It is also important because if you read the definition of DevOps, it is also the duty of DevOps engineer to measure the matrices and performance of the application. And try to make it better.
Lean Management
Lean Manufacturing or Lean Production, often simply “lean” is a systematic method for waste minimization into the production system without sacrificing system productivity. Japanese are very progressive to make things productive. Lean technique was basically introduced from the Toyota Production Company. The main purpose of Lean is to make things more productive and in less time and reduce the waste from the product.
Let’s we discuss some of the techniques followed by Toyota Productions to make things more productive.
Andon Card
It is a Lean manufacturing principal. In this technique, what happens is, in Toyota productions, if a single employee feels any kind of bug or waste in the production, then he tries to remove that bug and if there are chances that this waste can move into the product. He/She pulls the andon card. After pulling andon card, complete production stops and then all the factory tries to remove that severe issue which can be very harmful for the company repute.
According to Toyota Finance, after pulling the andon card company losses $1 million. But if that waste or bug moves into production and into products, then company may loss hundreds of million dollars. That’s why Andon card technique is very important to make sure about the quality of the products.
Blameless Postmortem
System always mature after continuous failures. If you want to make things more productive and powerful, then it is necessary to face the failures. Because failures help us to learn more and to do more. Men are not wrong but processes are wrong. In Lean management, one of the best practices is if there is any kind of failure that occurs in the system due to someone; instead of blaming him and demotivate him, the complete team does the blameless postmortem of the incident. They log everything that happens and how it happens and then all of the team learns from this incident. This is how they respect each other and make the team environment much better.
Kaizen (Continuous Improvement)
In this technique, teams are mostly focusing on different kind of experimentations and learnings. They share the ideas and problems with each other. They are actively participating in the things which are working and which are not working for them in their scenarios. This is how they master in the skills by picking up and trying the new things.
Lean Management is strictly following in the Toyota Production System but now it is also implementing in the Software Industry. If you want to know how we can implement the Lean techniques; here is a detailed book for you (Lean Software Development (Agile Toolkit)).
Blue / Green Deployment
If you read the different types of deployment models, there is one of the ways which is Blue / Green deployment model.
Actually, what happens in our real life, different teams are working on the same product. And this is the modern era in which different batches of commits or I would say different releases of software happens in a single day. Then, it would be very difficult to track which changes affect our complete system. So there is a model of Blue/Green Model.
In this model, what happens is, we have two types of servers. 1 is live server (blue) and the 2nd is staging server (green). Initially, traffic goes to the live servers and when we need to release a bunch of features, then we deploy those new feature changes to our green servers.
Image: Blue Green Deployment
And then, we run the smoke tests to the green servers to check if everything is working fine. And when all the tests pass, we route our traffic to our green servers. And when there is any kind of error in the green servers, traffic will be routed back to the blue servers. This is how we update our changes in the application servers.
Underlying persistence layer (database) is the same behind both of the servers. This is how blue green deployment works and helps us to track if there is any kind of error in the production and we can easily revert our changes.
Chaos Monkey
As we already know, it is also the responsibility of DevOps engineer to keep track the performance of the system and measure the matrices about uptime and downtime of the system.
Netflix is a media service provider whose complete system is based on modern technologies. It is using AWS (Amazon Web Services) cloud to provide the media services to our customers.
So you might be thinking what is Chaos Monkey?
Image: Chaos Monkey Quora
Netflix engineers wrote the script which is used to affect the production servers. This code is open source and available on github. Now when the chaos monkey is affecting the servers, Netflix Engineers fight against this monkey disasters in the production and keep the servers up and running. Chaos Monkey randomly shut down the servers. This is how Netflix Engineers test the system from unexpected outages and unexpected failure conditions.
As I have mentioned above in this article, if we want to make the system successful, then we have to face failures and fight against those failures. And if we are not facing the problems then there is very little chance to be successful. Men are not wrong but processes are wrong. So we have to make our processes strong.
Automation
As we have already discussed, DevOps is the name of delivery of valuable products to the customers very quickly. And if we want to do things fast; then we have to automate our processes. And here, the Automation comes into play.
With the help of automation, we can automate things. Here, we have some examples of automation:
- Infrastructure as a Code (IaaC)
- Automated Testing (Unit Testing)
Infrastructure as a Code
IaaC is a technique through which we can provision the new resources programmatically and not even we can provision the new resources but we can also maintain and configure the new resources.
If you have some experience with any Cloud (Azure, AWS), when we are creating the resources, then we can see the IaaC templates that are used to provision the resources. Like Azure has ARM and AWS has its own. So these JSON templates are used to provision the new resources. This is how IaaC is used to help us in our real life.
When we are working with DevOps, we need to use the servers at different point of times and you might have experience when we are working with new servers and deploy our application. Most of the time, our application is broken down because our newly acquired servers have different configurations. If there are few number of servers like 2 or 3, then we can configure them manually but suppose you’re working enterprise application and we have to configure hundreds and thousands of servers at a time. So the only option we have is to use the IaaC approach to configure all the servers within a few minutes.
Cheff, Puppet, Ansible are examples of IaaC automation and they are mostly used for the configuration management. We’ll explore all these tools in our upcoming series.
Automated Testing
So automated testing is the second name of unit testing. When we are following the DevOps coding practice, we have to write the unit tests. We commit the code with its proper unit test cases. And then in DevOps pipelines, our tests run and make the build. This is how we automatically test our application with different kind of scenarios. We can write the Unit test and we can also write the behavior driven tests of our application.
This is how automation helps us in DevOps.
CI Toolchain
If we discuss the open source tools of DevOps, there are many tools in the market. And different organizations are using different tools and even some of the organizations wrote their own tools to implement different kind of operations.
If we talk about the dashboards or boards where we manage the sprints and back logs of the our project, then we have the best option JIRA and Trello. And if we talk about running the test and making the build of our software product, then we will have the best option of Jenkins which is famous and popular in the market. If we talk about version control systems, then there is no competitor of Github which is used to maintain the public repositories and if want to maintain the on-premises source control, then we have the options of GitLab and BitBucket. And if we want to deploy the feature product, then we have the option of XebiaLabs Deploy and Octopus Deploy.
There are many other tools which I’ve not mentioned yet. And all these tools are amazing.
Azure DevOps
Azure DevOps is a SaaS (Software as a Service) kind of thing provided by Microsoft. All the above features like boards, testing, version control, deployment, and making artifacts of software product. We can do everything with Azure DevOps. Azure DevOps is the collection of all kinds of these features.
Azure DevOps Boards where you can collaborate with each other on the work that needs to be done to create the software product. Boards are focusing on providing a collaboration environment that fits the need of the team that is working according to an Agile and DevOps way of work.
Azure DevOps Repos are used for the purpose of source control. Here, we have 2 options to use the Git and TFVC. And we can commit and track all our changes to Repos.
Azure DevOps Pipeline provides us the way to do the CI builds for any platforms and low level language. It provides a web interface where we can create jobs to create production artifacts for your software.
There are two kinds of pipelines.
- Build pipelines: which are used to build the software product.
- Deployment pipeline: which is used to deploy the software to any target platform (GCP, AWS, Azure)
Azure DevOps Test Plans provides you the way to fully integrate your tests into your agile way. We can track requirements on the tests. We can run exploratory tests against the specific deployment that comes from the deployment pipelines.
Azure DevOps Artifacts are just like the version of software. You can upgrade the artifact with its latest version which is built by the pipeline and if you find any kind of defect, then you can easily revert it back to the previous artifact.
Another advantage of artifact is suppose you’re using any node package in your software and this node package has some kind of bug inside which is affecting some of our product features. So if we delete the package from the artifact and next time when we’ll install it again, then artifact automatically takes care of this and downloads and installs the previous version of node package which we installed before. That’s the beauty of DevOps Artifacts.
Note
The most beautiful thing of Azure DevOps is that we can use Azure DevOps to maintain our product and we can even deploy our application on any platform like (Azure, AWS, GCP, IBM) etc. It is totally dependent upon the client choice. And if we want only few tools in this complete suit of DevOps tools, then you can use that specific tool only and only pay what you use.
Conclusion
DevOps team tries to automate everything from testing of new code to how infrastructure in provision. In DevOps practice, we write the software in small chunks instead of going live in one go with all the features. In Traditional approach, we write bulk of code and consume months of testing on it. Writing small chunks of code allow us to increase the frequency of deployment and improve the time to deploy new code. It also enables us to iteratively process to monitor and measure to improve the code and operations every day. Instead of manually configuring software and hardware resources, we can use the DevOps tools to configure our systems within few minutes. This is how we can save a large amount time of teams in the company. With the help of DevOps, we are mainly focusing on the business instead of doing all the things.
DevOps is a new philosophy that can help software organizations innovate faster and be more responsible to business needs. It promotes collaboration between developers and operations which improves the software quality and more frequent software releases.
History
- 9th April, 2019: Initial version
