Why would anyone do something as absurd as mob programming?!
On a warm Spring afternoon, Alice, Bob, Carlos, Dave, and I were quietly enjoying a coffee break when I uttered a question I assumed I would regret: “Seriously, people, what are your thoughts on mob programming? Why aren’t we doing it?”
The looks on their faces spoke volumes. Bob was appalled and thought, “I thought this guy knew a thing or two about parallelism and resource optimization…”. Alice, a highly productive senior engineer, was disgusted, thinking about what a waste it was for five engineers to work simultaneously on only one task. Carlos, the youngest of the bunch, had a curious look. “I wonder if that cute girl from marketing is seeing someone?” he pondered.
Dave, our principal engineer, and an omniscient being, in his infinite wisdom, proclaimed that we would try it out. We were already a high-performing team and would deliver everything we committed on time, even if we experimented with this mob programming thing I kept whining about. The biggest benefit would be that they would get me to shut up about it finally.
A perfect chance arises
A software development approach where the whole team works together on the same thing, at the same time, in the same space, and at the same computer. This is similar to Pair Programming, where two people sit at the same computer and collaborate on the same code at the same time. However, with Software Teaming, we extend the collaboration to everyone on the team while still using a single computer for writing the code and doing other work.”
The next day, during sprint planning, we found a task that would be a good chance to try that approach out.
We needed to introduce a new authorization mechanism on a database gateway microservice. This microservice abstracts the massive database infrastructure away from developers of other microservices. It allows them to invoke stored procedures through a simple remote procedure call (RPC) without the hassle of establishing and maintaining database connections, preparing queries, and processing results, performing health checks and failovers when databases are in maintenance, picking a database instance with the fastest response and so on.
Since it is a high throughput service with many existing clients, backward compatibility, performance, and reliability were top priorities.
We estimated that it would take a single person about a week and a half to implement it. We would try to do it in a mob and see how it goes. We will all go into a room, look at the code on the big screen, discuss it, and code together.
Five rules for mob programming
On Wednesday, we gathered in a meeting room with a giant TV on the wall. I got up and wrote the rules on the blackboard:
- No distractions! Ignore Slack, email, and non-critical alerts. We need everyone’s complete and undivided attention.
- One person at a time writes the code. We take turns at the keyboard, rotating every 20 minutes.
- The main responsibility of the person at the keyboard is to listen and write the code.,
- People away from the keyboard discuss and navigate the person at the keyboard.
- We take regular breaks every two hours.
We had a brief discussion about how we were to proceed. Alice gathered the requirements in advance and did a little design up front, and Dave, the principal, gave us some pointers and advice before the higher-ups summoned him to join the other Avengers and help save humanity from imminent disaster.
First comes awkwardness, then comes progress
Bob was the first at the keyboard, and he obviously had stage fright as he constantly missed keystrokes and misspelled words. It was amusing to watch him struggle until it was my turn at the keyboard; I was even worse. Alice and Carlos were also nervous. It is curious how easy it is to reason about the code when looking at it on a big screen as someone else is writing it.
I tried to think about implementation and design while I was at the keyboard, but it quickly became apparent that my role was to listen and write. To obey the rules written on the blackboard.
The first rotation was stressful for everyone involved, but we pulled through. After a couple of rounds, the mood changed, everyone felt comfortable with the keyboard in hand, and then the magic started to happen.
“I’m supposed to change this part of the code? I have no idea what it does or how it works?” Carlos complained. Bob, who was familiar with said code, gave a short lecture and answered follow-up questions. Now Carlos is confidently modifying it.
“How will we ever implement this? It is too much work…” two people with little experience with this project asked; we pointed them to the modules that can be reused.
“Wow, how did you do that?!” – Alice is impressed with a clever IDE trick; minutes later, at the keyboard, she yawns as she performs the same trick for the second time.
“I have an idea how we can get some data we need without relying on the client to provide it. Let me show you before we go on a break.” – Bob, who has been around and seen the world shows us how, and we are all pleased with the proposal. We have routinely done it sub-optimally for years.
During our coffee break, we reflect on the experience. Two benefits we’ve seen so far: All of us are learning something, and it is a lot of fun. We are the kind of team that likes to hang out together, and we just found a new way to do it.
We are a little worried that we are moving too slowly. There are other tasks in the sprint waiting, and Mallory, the product manager, gives us the stink eye as he passes by.
On Thursday morning, we are back in the office, in a meeting room, ready to continue. We are steadily progressing, but we figured out that 20 minutes at the keyboard is too short, so we prolonged it to 30. We are constantly discussing naming conventions and design; we are continuously refactoring.
It becomes ever more obvious that the teammates are the customers for the code and that coding together is like having those customers on-site. You should write the code as if you are trying to get people to pay you for the luxury of reading and maintaining it.
The highs, the lows, the rollercoaster ride
As we continue to code, someone suggests we approach the part of the code following the Test-Driven Development approach.
We all write tests, some of us after implementation, some before. So, naturally, Alice, who writes them after the implementation, rolls her eyes as she grabs the keyboard and proceeds to write the first failing test. We hit our first serious obstacles that day.
None of us liked the metrics code around the cache we added, but we had no idea how to write it cleaner, and Dave was still out there fighting Thanos. Another issue was with a part of the code which performs a remote procedure call to our authorization server. It behaved strangely; we did not understand our RPC framework as well as we had thought.
After a frustrating half an hour, we decided to stop for the day – bumping heads against the wall just leads to headaches. Before our next session, one person will investigate how to do the metrics part better, and another will learn how our RPC framework handles exceptions in asynchronous calls.
On Friday, we were all working from home; we joined the call and shared the screen. The obstacles of yesterday were quickly resolved, and in a couple of hours, we had a working solution – something that works both locally and in our integration environment. We started polishing it up.
Bob proposed a simple refactoring, Carlos saw no point in it, but we still did it. When it was done, it was obvious why Bob was keen on doing it. I wondered how long it would take to explain the same idea through the comments on the pull request. We started cleaning up the code, following the adage, “First make it work, then make it right.”
And as often happens with this kind of work, every modest improvement makes way for another unforeseen. As the code was getting cleaner, it was easier to spot mistakes and opportunities for improvement.
When the new code is a mess and schedules are tight, there is always the temptation to check if the code works say, “It looks good to me,” and approve the pull request. In this industry, that is known to be a typical prologue of many really scary horror stories. Programming in a mob helped us avoid that pitfall.
On Monday, we decided to call it quits; Bob would do the last 5 percent of the work alone – add some logging and documentation, write a couple more tests, and deploy it to production.
We all agreed that our code was far superior to anything we would have done individually. A few thousand lines of code everyone is satisfied with and understands used to sound like science fiction – now it was reality.
When we add up the person-hours it took to deliver the feature – we were slightly faster as a mob than when using feature branches and PRs.
Our engineering manager put it well:
In programming, you do 90 percent of the stuff smoothly, then you get stuck on some stupid thing and lose a lot of time figuring it out. When you’re in a mob, chances are that someone has already bumped into this stupid thing before or can quickly spot a bug you accidentally introduced.
We now have a complex feature that is thoroughly understood, can be easily improved, and quickly be troubleshot by everyone on the team. There was no waiting on PR approval; we all learned new concepts and keyboard shortcuts, tried different coding styles, and saw how the interplay of different thought processes plays out. We practiced talking accurately and listening carefully, we delivered the feature, and we had fun.
The unexpected benefits
I believe the main reason our sessions were successful is our team’s values of communication and respect. Psychological factors were also at play: we are not afraid to make mistakes and look stupid in front of one another, we cherish the opportunities to share knowledge, and there are no “my way or the highway “egomaniacs on the team.
Company support also plays a part; there is no micromanagement, and if we are steadily delivering value, no one cares how we do it; it is up to us to self-organize.
The team mindset about mob programming changed dramatically since a little experience can quelch many common-sense prejudices. Common sense is the great deceiver. We have been doing an excellent job so far, but we did not want that to be an obstacle to improvement. We agreed to continue the old way for simple tasks that can be quickly implemented and reviewed; for the more complex stuff, we will consider mob programming.
“You’re not coughing anymore, and you look kind of cheerful. Is everything okay?” – Dinko from human resources asked me.
“I finally got the team to try mob programming.” I smiled as I explained what it was.
Dinko was amused, “It seems to have some health benefits as well; you should write an article about it. It might even end up on ShiftMag,” – he said.