What Is DevOps?

By now you’ve probably heard the term “DevOps” (before downloading this book, of course). The word DevOps is a combination of the words “development” and “operations,” which gives you an idea of what DevOps entails. DevOps is a set of practices that focus on collaboration between developers and operations (or system administrators). This may also include testers and security, and basically any party that is involved in delivering software to customers. The goal of DevOps is to release more-reliable and better-quality software at a faster and consistent pace.

Azure DevOps used to be named Visual Studio Team Services (VSTS) and Visual Studio Online (VSO), and is the cloud successor of Microsoft Team Foundation Server (TFS), now named Azure DevOps Server. Azure DevOps helps teams by implementing the technical side of DevOps. In this chapter we’re going to explore DevOps, what it is, and how it can help companies produce better software, faster.

The history of DevOps

The term DevOps was first coined over a decade ago. Back then, different teams were often completely separated from one another. Developers would write software and hand it over to testers, who would test the software and either give the results back to the developers, or give a go to operations. Operations would then deploy the software.

It was not uncommon for these teams to be at separate locations and have no contact with each other—one team would do their job, and then throw it over the fence. The different teams would often have varying, conflicting goals and metrics of success. For example, developers need to deliver a feature as fast as possible, testers need to find as many defects as possible, and operations needs to have as much uptime as possible (and an update requires downtime). The result, of course, was that software was often buggy and delayed, and teams were unhappy and frustrated.

With new technologies such as virtualization, cloud, and containerization, as well as various automation tools on all levels of the development lifecycle, the expectations of both IT professionals and customers changed. Everything must work seamlessly, without downtime, and on demand. For developers, this means they need instant access to the various resources needed to run their code. Back in the day, this was not possible, as various teams were separate entities who protected their own domains. The result was unhappy customers.

The first DevOps conference, named devopsdays, was held in Ghent, Belgium in 2009. The first “State of DevOps” report was published in 2012. The report has been published yearly since 2014, when DevOps adoption started accelerating. DevOps is now a trend in the industry, inspiring conferences around the world. With DevOps practices and the right supporting tools, teams can deliver value to customers in a matter of minutes. A change request from the business can be in production minutes after a developer checks in the code. When fully realized, DevOps enables non-disruptive innovation at scale, anywhere in the world.

The basics of DevOps

As you may have concluded from the history of DevOps, it’s not so much about tooling as it is about company culture. One where everyone works together to bring value to customers. In the broadest sense, it is the process of improving coordination and collaboration to produce better, more reliable products, and to achieve business goals more quickly and reliably.

By practicing DevOps, companies improve collaboration between teams, but the teams change as well. Instead of having teams defined by the work that they do, teams are defined by the result they deliver. So instead of having a team of developers and a team of testers, each team now has a developer and a tester, because they are both necessary to achieve the goal. Next to developers and testers, teams often comprise a Scrum master, product owner, business analyst, system administrator, and security engineer. Teams can change and people can be on multiple teams.

By adopting Agile methodologies, such as Scrum, teams can deliver value even more quickly, and at the same time increase stability and quality. The basic idea behind Scrum is that small teams of about four to seven people work in sprints of two or three weeks. During a daily Scrum meeting of about 15 minutes, they discuss their progress. A few times a week, the team takes time to look at open user stories and tasks that need to be carried out, estimate whether the story is clear enough to be carried out, and assess the amount of work necessary to perform the task. A product owner sorts the stories by priority, and because a team knows how much work they can do per sprint, the top x stories can be planned for the next sprint. At the end of each sprint, the teams do a review of the sprint to see what went well and what could have been done better. This way, teams continually strive to improve themselves.

Figure 1: The Scrum Lifecycle. Sourced from Microsoft Azure documentation.

This is a paradigm shift from the old Waterfall method, where specifications were collected up front and passed on to developers, who would then write code for months or even years straight, before ever delivering anything. When the team finally delivered, the specifications were often outdated and the software did not satisfy current needs.

There are other Agile methodologies, like Kanban, but Scrum is one of the most popular today. Like DevOps, these methodologies aim to deliver higher-quality software faster, and at a predictable pace. Agile methodologies and DevOps complement each other. Where Scrum can be an implementation of the cultural aspect of DevOps, DevOps tooling has a focus on automated software builds, tests, and deployment. And that tooling is where Azure DevOps comes into play, as we’ll see in the remainder of this book.

Automation

While a certain company culture is required for DevOps to succeed, many people focus on the technical aspect of DevOps. The key word here is automation. When a developer and a system administrator work together, it suddenly becomes possible for code to be shipped automatically, using automated tools that are already at the developer’s disposal. That’s where Azure DevOps comes in, but other tools like GitHub, GitLab, Jenkins, and Bamboo can do the same thing.

Imagine that your code is under Git source control. The moment you check in new code, a build is automatically triggered on the server. When the build passes, you’re sure you didn’t commit any typing errors, omit references to local packages, or anything else that would prevent your coworkers from building the source locally. After the build is passed, your code is automatically tested using the unit tests you, your coworkers, and your testers wrote. Once those tests are passed, your code is deployed to a staging environment where testers can do some manual tests, like acceptance tests. After the testers give their blessing, you can one-click deploy the software to a production environment with little or no downtime.

The process of automatically building and testing your code is called continuous integration, or CI for short. The automatic deployment after that is called continuous delivery. If all manual steps are eliminated and the code goes from a commit to production fully automated, we’re talking about continuous deployment. Both continuous delivery and deployment are abbreviated as CD. In practice, most people mean continuous delivery when talking about CD. Very few companies feel comfortable with the thought that code goes into production completely automatically. You will often see the term CI/CD, meaning teams do both, although you can’t have CD without CI. CI/CD is also sometimes used interchangeably with DevOps.

With tooling such as automated builds and releases, version control and work tracking, teams have everything they need to collaborate on software projects. Azure DevOps has support for all of them. Where other tools do one or two of those things, Azure DevOps is a complete solution.

The cloud

With the emergence of cloud technologies in the past few years, DevOps has become increasingly more popular, especially CI/CD. It’s never been easier to deploy various resources using just a script. Whether that’s JSON, PowerShell, or a CLI script, it’s now possible to deploy a brand-new website with little or no manual intervention.

In fact, Azure DevOps itself is a cloud solution. It’s the successor to the on-premises Team Foundation Server (TFS). As can be expected from a Microsoft product, Azure DevOps has out-of-the-box integration with .NET, .NET Core, and the Azure cloud. In the next chapters we’re going to create an Azure account and deploy a website there, but we’ll also deploy locally to our own computer. In the cloud, however, we see development and operations really coming together. Gone are the days when you needed physical access to a server to deploy software on it. And since everything can be deployed using scripts, developers can write the deployment procedures. Operations, or system administrators, can help in this endeavor, but may feel more comfortable in providing access and monitoring cloud resources. Of course, when it comes to networking, like limiting access to your public cloud to your own internal network, it becomes apparent how teams with both Dev and Ops can really work together.

With both cloud and DevOps, new architectural patterns have become possible, most notably microservices. With microservices, an otherwise large application is divided into a set of smaller subsystems where each subsystem can function completely on its own. For example, an application can have a product module, a sales module, a stock module, and an invoice or financial module. Going into the details of microservices is outside the scope of this book, but it is important to note that these services all have their own databases and possibly run on different servers. The advantage to having microservices is that scaling is easier than with one large application, or a monolith. Especially when done in the cloud, scaling becomes massive and automatic. Also, when one service fails, parts of the system can still be used if other services continue to work. The downside is that you have a lot of deployments. Instead of deploying one huge application (which is usually just a few files anyway), you now have to deal with tens or even hundreds of releases. Releasing everything manually would be a full-time job, but with CI/CD this process is completely automated. Containerization tools such as Docker and Kubernetes help to manage these complex deployment scenarios.

Culture

We have already been over this, but I can’t stress this enough: DevOps is about company culture. However, forming cross-functional teams and calling yourself Agile is not going to cut it. That is something I come across often—companies calling themselves Agile while they’re just unorganized, or saying they do DevOps while they are just running a CI pipeline. That is not DevOps. DevOps is about changing the company culture and having the tools to support that culture.

When teams are cross-functional, they have to take shared ownership of the code and the environment. Usually, when a bug is found in production, finger-pointing begins. The developers will blame the testers, testers will blame developers or system administrators, and system administrators will blame everyone else. After a team takes shared ownership, it is not anyone’s fault; it is the team’s fault, and the team will have to solve the issue. It is a shared responsibility. And suddenly there are fewer conflicting interests.

In DevOps it is not important who gets the blame. It is important that issues get fixed as soon as possible, even when the original author has a day off.

Of course, this change in culture does not stop with the technical team. Bugs in software are often blamed on users asking for impossible features, or on management cutting time or budgets. However, blaming is not part of DevOps. If users and management are involved in the development process, impossible features can be discussed with users earlier, and management knows about time and money constraints before they result in bugs and unhappy users.

The mindset in a DevOps environment should not be that of name and shame, but of blameless and collaborative problem solving. By adopting Agile methodologies with cross-functional teams, everyone is involved in estimating and planning work.

Where does Azure DevOps fit into this?

Azure DevOps supports teams that work in a DevOps environment. DevOps allows companies to create teams and projects and let them work independently of each other. With Azure Boards, teams can create and prioritize user stories, estimate efforts, and plan sprints. Repos allows teams to put code into Git source control, link commits to user stories, and enforce code reviews. By linking commits to stories, you will more easily be able to see how far a user story is completed or why bugs happen.

Pipelines are an important aspect of Azure DevOps. In fact, I have worked in teams that only used this feature. With pipelines, teams can create CI/CD pipelines. It is easy to automatically trigger builds or releases on code commits.

In a build pipeline, code can be pulled from Azure Repos, GitHub, Bitbucket Cloud, other Git sources, and even Subversion. You can build, test, and package code in pretty much every language. Needless to say, .NET and .NET Core are supported out of the box. You can also build Java projects using Maven or Apache Ant, or build Go applications. Tools such as Grunt, Gulp, and Docker are also directly available. For JavaScript projects, there’s npm support. Even Apple is supported with Xcode and Xamarin.iOS (as well as Xamarin.Android, of course). Your builds can run on Windows, Ubuntu, or MacOS, but you can use your own device as well. Furthermore, pretty much anything is possible using the Bash shell, the Windows command line, or a Python script.

Once your build is finished, you can deploy your code using a release pipeline. Your pipelines can have multiple stages, like development, test, and production. There is plenty of support for Azure deployments. You can also deploy databases, for example: SQL Server with a DACPAC or SQL script. You can deploy using Docker and Kubernetes. Even if something is not available in your pipelines, chances are you can add it from the Marketplace. For example, you can add SonarQube, Yarn, AWS Toolkit, Terraform, or Google Play.

A less-used feature of Azure DevOps is test plans. This is where testers can create manual test plans, run them, and create test reports. Tests can be created for specific versions of your software or for sprints.

Last, Azure DevOps offers artifacts. These are reusable software packages. Supported package management tools include NuGet, npm, Maven, and pip.

Summary

DevOps is a mindset that is all about collaboration and shipping high-quality software at a predictable pace. Azure DevOps offers a complete package for developers, operations, and business users to support collaboration and automation. In the remainder of this book, we are going to explore the various features Azure DevOps has to offer.