Welcome to another episode of The DevHops Podcast! This week we’re joined by Gary Gruver, co-author of Leading the Transformation: Applying Agile and DevOps Principles at Scale. It was great to have Gary on the show, as he provided a wealth of info around DevOps at the enterprise level and how organizations can truly set themselves up for success as their initiatives expand across multiple services.
For those who don’t already have a copy of Gary and Tommy Mouser’s new book, we can’t recommend it enough, and if you sign up for Gary’s newsletter at www.GaryGruver.com, you’ll get access to the first two chapters for free!
And now, on with the show!
Can’t listen right now? No problem! Below is a transcript of the entire episode for your reading convenience!
Noel: I really enjoyed your book, Gary, and I think it’s come out at a really great time. We’ve turned the corner on not having to spend so much time defining DevOps, and we’ve moved on to talking about how to scale it because so many people have already begun their journey or transformation. Early in the book, you say that “Continuous delivery tends to cover all of the technical approaches for improving releases, and DevOps tends to be focused on the cultural changes.”
There was so much confusion as to what DevOps was even just a year ago, but I think that separating the culture from the technology is a great approach to eliminating that confusion and knowing what’s going to be required to implement DevOps successfully.
Gary: I’ve evolved a little bit in my thinking. When I’ve heard Gene speaking lately, he’s really trying to say that DevOps is all the technical, cultural, and practices that it would take to enable a company to release code on a more frequent basis while maintaining quality, stability, reliability—all the “ilities.” That captures it all because you can’t do one without the other. Even if you do all the technical changes, but you don’t get the cultural changes, your benefits are going to be fairly limited.
Noel: That’s true. You advise in your book, applying DevOps principles at scale for enterprise solutions is going to require implementing continuous delivery. I wanted to know why that is. Not that I disagree, I just want to understand more about why continuous delivery is going to have to come at some point on this journey.
Gary: As you’re trying to release on a more frequent basis, what you’re really trying to do is integrate all the work in the organization and put it together and make sure that it’s working on an ongoing basis.
Continuous delivery is the technical approaches that make that happen. It starts with continuously integrating all your code, testing it, and then it dealing with all the issues that you need to resolve to make sure your environments are consistent and giving you the same answers as code moves down your deployment pipeline from a developer’s desktop to end of production.
The thinking is that, if you don’t do the scripted environment and scripted deployment part of the equation, the feedback that you’re giving to the developers is not going to be the same when they’re checking in their code as it is when you’re trying to get it into production.
You’re just trying to resolve and deal with all those differences and make sure that you’re consistent throughout the path. What you’re trying to do is make sure that when a developer checks in their code, and it passes that first gate, that, that code will work all the way out in production without finding new and unique issues—which tends to happen if you let environment or deployment differences creep into your system.
Noel: I was at a testing conference last year, and Adam Auerbach from Capital One presented a session where he recommended something that I hadn’t heard before. He was talking about how they were doing DevOps and that they were working towards continuous delivery, but he showed all these amazing accomplishments and progress that they had made. Then someone said something about trying to get continuous delivery themselves, if this is what it looked like. He said, “Oh no, this isn’t continuous delivery, it’s close but we’re not there yet.” He said, “And for you, I would advise for you to begin doing these things as well because you may not actually get to continuous delivery, but that getting close still gives you a ton of rewards,”
I wanted to get your opinion on, if getting close can be considered still good enough or if you really do need to do everything you can to actually make it to what continuous delivery actually gives you.
Gary: As I talk with Jez, he makes a very clear distinction between continuous delivery and continuous deployment. Continuous delivery is all the discipline and rigor around putting everything under version control whether it’s the code, the tests, the environment definition, the deployment process. That rigor of putting those things under revision control and automating them to where they’re reliable and consistent across all the different stages of your deployment pipeline, that is what I think of as continuous delivery.
Continuous deployment is a business decision that says, “Okay, I got all that structure in place. I’m no longer constrained by the ability to deploy code in the production by my development process. Now I need to decide whether I want to do it with every check-in or not.” In a lot of cases, some businesses, that wouldn’t make sense and it wouldn’t be a value. Especially if you’re doing embedded things, where you’re not in control of the deployment.
I would agree with him wholeheartedly that as you start to automate your environments and get consistency across them, and you start to automate your deployments and get consistencies across them—as you do the same thing with testing—you are going to be solving issues that your organization has been struggling with for decades. What you’re doing is increasing the frequency of those things to the point where you can no longer brute force your way through, issues that have been plaguing you for a long time, and you have to start fixing those things with automation.
So, yes, the closer you get towards it, the better off you’re going to be and you may never get to the situation where your organization needs you to do continuous deployment.
Noel: That reminds me, in your book, you talk about how automating some of the processes you just mentioned, enterprises can “dramatically improve productivity.”
Gary: Yeah, and you start by taking on the things that are causing you the most pain. If you used to do a build a week, or a build a day, and you start trying to do ten builds a day, what are the things that are most painful? Those things have been painful for the organization all along and it’s just when the deployment frequency was way down, you were able to brute force your way through it. As you start to increase that frequency, you can no longer use brute force to get through it and you need to start fundamentally fixing those issues that have been plaguing your organization for years.
Noel: The last question that I had for you, you talk about how the need to determine if your organization will embrace these cultural changes up-front is really important. If they won’t, you say there’s no sense in making big investments in the technical solutions that on their own won’t help. Like you said earlier, that if you just do the technical aspect without the culture, you’re not going to get there.
I wanted to see if you had any advice as to how do you determine whether you’re going to be able to get that buy-in early, and maybe if it looks like it’s going to be difficult rather than just assuming that it’s not possible and throwing in the towel—if there are some ways to get that early buy-in so that you can begin this transformation.
Gary: Yeah, I think one of the first places I started, if you look at my book, we talk about forming a team to lead the continuous improvement process.
Getting the executives aligned on leading the transformation. If you’ve got the leader of QA, and the leader of Dev, and the leader of Ops, and the leader of security all driving on different sets of objectives—it’s going to be hard to get the teams to come together to align on delivering improvements and what to improve, and how to improve.
The first step is, can you get those leaders to come together and start to lead monthly checkpoints where they agree on what we’re going to fix and we set that in place? These are the objectives everyone is going to drive to achieve over the next month. Then the job of the executives and the management team is to monitor the progress and play the role of an investigative reporter in the organization trying to figure out what got done, what didn’t get done, and what did you learn so that they can set the objectives for the next month.
If you can’t get that aligned, I think that’s where you should spend your energy and your effort and maybe you need to take it up in the organization and bring the book to a higher-level executive over that to see if you can that alignment or bring a consultant like myself to work with the team to see what’s in the way of them coming together and agreeing on those common objectives. That’s one of the big cultural changes, is getting that team aligned and engaged in leading the transformation. One of the reasons I wrote the book is, I think that’s one of the biggest holes in organizations is making that happen.
Number two is, can you get the Dev organization to respond to the feedback that you’re designing and building in on the deployment pipeline? If they want to go off and just develop code and not make sure it’s working in an operational environment and not prioritize making sure that’s going to work, then you’re going to get limited benefits.
Set up a simple CI environment and get them in the cultural process of keeping a few automated tests running. It doesn’t have to be every test automated and it doesn’t have to be the entire enterprises pipeline stood up, but can you get a simple CI environment that they’re responsive to, and they’re reactive to, and the work ventures out. If you can do that, then you’ve started down that cultural transformation.
The next one is, can you make sure that the tools that you’re going to use to deploy and define the environment in production are the same that the developers are using. If you’ve got developers using completely different tools to define their environments and deploy their code, then you’re going to have a disconnect when you go into production. Can you get the operations teams and the development teams to come together and start working on them? I think those are three key first things that you can start to work on and get going.
The book has some other transformational spots, but don’t go off and completely automate everything and don’t go off and wait until you’ve completely scripted the environments. Start using the test automation and things that you’re developing as soon as you possibly can and get the organization coming together and aligned on going down on a journey and a path together.
Noel: That’s really great advice. I’ve talked to some who have led successful DevOps initiatives and there were, of course, some early pitfalls. A lot of them all talked about the need to establish that trust early and that’s done differently with each department. They’re all going to have certain areas where they’re going to need some trust that this is a good idea and that it’s going to improve the quality of the software, or the time to delivery, or just make their own lives better, but establishing that trust early and not just diving into it automating everything possible is a much better idea for getting their trust and buy-in early on.
Gary: Yeah. I think the key part with the executives is making sure you get their fingerprints on it. That it’s their plan that they believe will help meet their business objectives. If you can get them focused on their business objectives and how what you’re doing and the transformations you’re making is not doing DevOps to do DevOps, but using it to solve one of those problems you identified. Are you having quality problems? Are you wanting to release more frequently? Are you trying to improve your productivity? What are those things that you’re trying to do?
If you can’t get the executive teams to believe that it’s key to their business success then you are going to struggle. You need start by getting their fingerprints on the plans so they own its success and help lead the organization change. Another thing that I say is, get that aligned team to go down that path together because if you leave compliance or somebody out of it, they’re likely to throw rocks at the changes you’re making. If you start with a set of business objectives they can all agree to and you get their fingerprints on it, then it becomes their plan and they’ll make their plan successful.
This podcast is part of our continuing technical series, “Scaling Modern Software Delivery.” We encourage you to learn more about our unique perspectives around inefficiencies that have hurt software quality and time to market, and how organizations are removing these issues from their software delivery processes!