My philosophy on leading a software team goes something like this: Teams are empowered, and can be trusted to do the Right Thing when the Right Thing is also the path of least resistance.
From a managers perspective, this often means communication, tooling, budget, time, etc, but an architects can take a more technical, hands on approach. Let’s dive into some techniques that can help make it easier for your team to do the Right Thing.
Years ago, I used to think about this concept as Architectural Enforcement (I’m a hockey guy, what can I say), but a colleague of mine convinced me that Governance was a better term, even if it is more politically correct.
Architectural Governance is the use of tools and products to prevent programmers from ‘breaking the rules’ — or more appropriately, to cause a conversation to occur when breaking a rule might be ok.
Checkstyle and FindBugs are two examples here, and easy to implement on a new project – is it a pain in the ass when you commit code, and it fails to build in your pipeline because your if statement looks like this:
if(aThingIsTrue) instead of this:
if (aThingIsTrue)? Absolutely, but it’s hard to deny that it makes the code more readable, and over time, you’ll realize that you need to remember to actually build your code before pushing it to your shared repo, and no-one is going to argue that that’s a bad idea.
These tools are also pretty pliable, so if you find that a default rule doesn’t sit well with your team, just turn it off – this isn’t about forcing everyone into a set of practices that they don’t like, it’s about a team making a decision that there certain standards and guidelines that everyone should follow, and then putting the tools in place to actually make sure they’re followed.
Quick show of hands – how many of you have worked on a team or in a company that had a ‘Coding Style Guideline’ of some sort published on a team wiki, but a quick look at the code shows that it’s completely ignored? Think a bit about who is actually reading that wiki. In most cases, it’s going to be your new employees — do you really want to send the message to new employees in their first one or two days that the guidelines are just there to look pretty, but don’t worry about it, because no-one pays attention?
Other tools and concepts in this space include much of the Simian Army – what better way to ensure that your teams are properly handling failures by causing failures to occur. I’ve used simpler tools in the past like aspect oriented filters to ensure that a Controller only works with Services, and doesn’t directly access a Repository, or vice versa. There are plenty of techniques to use – using these Governance tools can help ensure that your teams know what the Right Thing is, and can also help identify when the Right Thing might just need a little tweaking.
The term ‘build pipeline’ has been popular the last few years, since the publishing of Continuous Delivery (if you haven’t read it, do it. Seriously, don’t wait – go do it!), but it’s not a new concept. Build Pipelines are simply Continuous Integration processes taken to their logical conclusion, and realizing that automating your Unit Tests or even your Regression tests don’t mean a lot if that code is left sitting around, or if it the automation stops before the software is actually deployed. What good are the hours spent testing, if the production software release process is completely different than the testing and preprod processes? Are you prepared to tell your QA team to go home, because the testing they’re doing won’t be valid when it’s deployed to production?
My full set of thoughts on this topic would make this post far too long, so for now a few principals will have to do:
- Start with a tool that allows you to put your configuration into Source Control – building your pipeline by hand on a project by project basis is a great way to make it unrepeatable. The good news is that there are plenty of tools that can do this – GitLab Runner, Travis CI, and yes, Jenkins 2.0 are just a few options.
- Build your artifacts once, and promote them as they move through the pipeline. This will help you keep track of what builds are ‘approved’, and will eliminate any chance that the thing you tested is not the thing that you released.
- If you have one team that develops the software, and another team that releases the software, it is wrong for either of these teams to build the pipelines in a vacuum. Release Pipelines are a phenomenal tool to help your development teams and IT teams work more collaboratively – the term is ‘DevOps’ for a reason, not ‘NoOps’!
- Optimize to fail fast. If you have some tests that take time to run, execute them last, so you don’t have to wait 15 minutes to discover that you missed that Checkstyle issue mentioned above.
- Categorize your tests stages, and build them up over time. This is about practicality – you should start by identifying the types of tests that you want in the pipeline (unit, integration, acceptance, performance, release tests, visual diff tests, etc), but if you refuse to use your pipeline until they’re all in place, you’ll never get there. Instead, configure your pipeline to run your build, unit tests and release processes, but leave the final release processes to be manually triggered stages, rather than automated. At the same time, make the decision about what tests you absolutely must have in place before you’re willing to automate that final release, and then add those steps to your project plan. This will give you many of the early benefits of a pipeline, and it will give you a controlled release process – you can then make it more efficient as you go.
- Recognize that 100%, hands off release automation is not necessarily the goal here. Having a controlled release process that is agreed upon by development, IT and QA is. If you still believe you need someone to hit the button before release, that’s fine, just recognize that each release will generally include larger change sets. (While you’re add it, find some internal tools, or less ‘mission critical’ apps, and automate the crap out of them – this will help you gain a bit more confidence with the process.)
Maven Archetypes were one of the really valuable concepts that Maven brought to the Java world, but one that I haven’t yet found a really compelling implementation of – as good as the dependency mechanism and Archetype concept of Maven, I never really like using Maven as a tool on a day to day basis. It had just as much of the XML ugliness as Ant, but because it was a declarative tool instead of a procedural tool, it always felt less readible.
The idea behind Archetypes was pretty simple, though – define a basic, empty structure of a ‘type’ of project, and provide the tool to allow a developer to recreate this in a single command. Pretty straight forward, although somewhat limited for the time – we were building a lot more monoliths than micro-services back then, so the need to create a project from scratch was pretty infrequent.
Micro-services is the latest hotness these days, but rapidly developing cohesive, yet independent systems across an organization is hard. Not only do you need to consider basic project structure, but you also need to worry about integrating with test frameworks, setting up circuit breakers, building a deployment pipeline, etc. Thinking in terms of archetypes can give teams a head start, and do it the Right Way.
Unfortunately, I don’t know of a tool that does this really well. Several tools will help move you in the right direction – tools like Gradle, Vagrant, and Serverless all have ‘init’ commands that will get you started, and the Spring Framework has the Spring Initializr, but none that I know of allow you to cleanly define your own (Gradle might be going in this direction with the InitBuild plugin, but it’s still in incubation and doesn’t have the option for a team to define their own project types).
So for the time being we might be stuck with ‘thinking’ in terms of Archetypes, but there’s still value here – creating a blank project template that defines folder structure, and includes configuration templates is easy, and can save a lot of time for teams that are building out their infrastructure. It also serves as an effective way to communicate what the current best practices are, as they are discovered and added to the templates.
Still, there might just be an opportunity here somewhere…
Of course there are more techniques here – this is only a start, and likely a topic that I’ll expand on in the future. The key is about thinking in terms of making the Right Thing the Easy Thing – if the process of pushing out a hot fix is exactly the same as pushing out any other release, and if that process can run start to finish quickly, you will find yourself doing crazy things like opening a .jar file and replacing a .class file far less frequently. Yes, many of us know how to do that, but most of us also know that it’s not ok.
BTW, if you couldn’t figure out the difference between those ‘if’ statements above, look for the space…