MuCon 2019 Takeaways

MuCon 2019 (or µCon) was hosted by Skillsmatter in London and is a conference for Domain-Driven Design and microservices. I was able to attend and I thought I would share my experience. Fervently writing notes during the sessions and now sharing them really helps to reflect on what was presented.

Summary of day one

The common thread throughout the day, and possibly the whole conference, was how team structure greatly influences our code and the other way around. We need to design our teams just as we do our code. We can also design our code to influence our team structure. Independence is important, equally when scaling code as when scaling teams. You can’t hope to scale code riddled with dependencies just as you can’t scale teams with dependencies.

Keynote: Socio-Technique and Structure

By Michael C. Feathers (video)

Team dynamics are just as important as how we structure our code. If we have direct in-process calls between components, maybe that’s how our team will look like and changing processes will be hard. If we have a service bus loosely coupling our services, maybe in the same way, it’s not unsurprising to see a single person sitting in between teams communicating what is required.

Michael talked about something that I’ve noticed as well. There is research showing that much of a code base is written by a smaller part of the team (see Why software needs heroes in the resources below). I don’t think we should see this as neither a good thing or as critique, simply as how things are. Maybe it’s even evidence of the efficacy of the surgical team as mentioned in the Mythical Man Month.

He referred to Galileo’s law and how structures need to have an architecture to support the volume it contains. I’ve failed to find a resource describing this but, intuitively it makes sense. A simple mobile app to re-colour pictures won’t need event sourcing and auditing whilst a banking application does. In other words the patterns we apply must be suited for the context.

Other good takeaways were that smaller teams and teams using the same processes and tools can far outperform large and divergent teams.

Microservices from Day One

By Richard Roger (video)

Richard shared his personal experiences building a system with microservices in his current startup. He also mentioned how they, after trying to use different languages in different services, went back to standardise. Turns out that using fewer languages still make sense. I’ve heard from several speakers now how it seems that despite microservices allows us to use different technologies, this often sounds better than it is.

The biggest takeaway was the importance of messages and pattern matching for decoupling. He had this very good diagram to illustrate how pattern matching can be used to act on different messages based on traits and how this allows us to decouple and reuse components. This is my poor attempt at reproducing it:

What Do You Mean I Can’t Google It? – Search for Microservices

By Heather Whyte (video)

Elastic search, which began as kind of a log aggregator, has now seemingly become an industry standard as a tool in implementing search engines. I’ve seen its use in this context many times now. Heather exemplified how they migrated their monolith application into a more flexible and extendible system using microservices.

The Gordian Knot

By Alberto Brandolini (video)

This was a first for me hearing about Alberto but he was brilliant. He was referenced time and time again by other speakers as well and I definitely recommend checking him out online.

He made it abundantly clear for me how breaking an application into microservices only makes sense if you actually have microservices that are independent. If they’re not independent, you’ll get into a distributed mess and be no better off.

He talked about “winning behaviour”, linking back quite a bit to Feather’s talk on team structure. He talked about how we can improve code structure by designing our interactions in such a way that good behaviour is rewarded instead of bad.

Maybe not in terms of rewards but, in terms of interaction, I’ve seen this happen in my current team. We had one big monolith application which we could not possibly all work on at the same time. We split it into two and effectively the team got split into two, naturally. The result, we can now independently deploy, upgrade and plan these two systems. This means less cognitive load and easier management, not to mention how much easier it is to develop.

He also pressed the point (which I heard several times again during the conference) about the importance of feedback. How, if you get quick feedback you’re prone to adjust but if you get either yelled at or encouraged weeks past achievement or mistake, chances are you won’t learn much.

Good developers are OK with paying for their own mistakes. As long as they get quick feedback.
– Alberto Brandolini

He suggests using two checks two measure how well designed your team interactions are, the Pink check (from Daniel H. Pink) and positive feedback loops. I might try to cover this in a separate post.

People won’t improve a system if they won’t stay around long enough to see the benefits.
Have you ever repainted a hotel room? – Alberto Brandolini

Strategic Domain-Driven Design Patterns

By Nick Tune (video)

How we work can be changed by the architecture. Basically, Conway’s law in reverse. We can actually force our behaviours through intentional design. Although it wasn’t mentioned as an example, I thought of how CI/CD has immensely changed how we work. Before, we didn’t have anyone stopping our PRs when test coverage or build failed. Now, with modern tools and automation, I strongly believe this move shapes the way we work and how organisations move towards faster delivery, from the bottom up.

Nick had this good radar diagram on how we need to choose patterns to apply to each solution based on several different factors:

Event Driven Collaboration

By Ian Cooper (video)

Data on the inside vs data on the outside. A clever way to divide what we should be able to change and know about and what not. Simply put, data that leaves your microservice (or context) shouldn’t be possible to change. Only one place should be able to change data, the single writer. This lives on the inside of your service, once it leaves your service that data is “dead”, it’s a memory of what the data looked like when it was there.

By sending events rather than requesting data (like the dead data above) we can assume that our current view of what’s real right now is up to date. This allows us to do something cool, to cache reference data.

Imagine we have a basket and when we add a product to the basket we want to validate the product exists. Querying the products to validate this for every insert, well, that can be time-consuming but if we instead get a notification (event) for every product added and cache this, it becomes simple. We can refer to this as reference data and we can cache it. Ian mentioned ways of using standard HTTP protocol, E-Tags and Expiration headers as well as the Atom protocol to implement faster ways of building these caches as well.

Keynote: Microservices & Containers: Getting your Security Team on Board

By Liz Rice (video)

This was a really great session on how the three jobs of security teams are affected by containerisation:

Patching vulnerabilities; scan the container images, if they raise issues, patch and rebuild. The tests will validate it works. Tools such as Microscanner can help.
Ensuring network security; tools such as kube-bench can automatically help find issues with your meshes. Infrastructure in code and automation actually allows for a more fine-grained level of detail than what is plausible with manual configuration.
Zero-day attacks and intrusions; Doesn’t change does it? In fact, configuring checks on all containers can likely be done much more generic than applying it per server in a wide landscape of different server types.

In summary, containerisation isn’t the enemy but there is a need for a new kind of DevSecOps movement which embraces automation and collaboration with DevOps.

Summary of day two

Building things piece by piece and focusing on what is important. It sounds easy but we often forget what “important” is. Many of the talks, not just today, touched on the importance of weighing your options. Maybe, possibly even likely, microservices won’t solve all your problems. There are no silver bullets and your context must drive your decisions.

Keynote: Getting to DDD: Pragmatic or Principled?

By Julie Lerman (video)

Try memorising this:

From http://domainlanguage.com/ddd/reference/

Impossible right? That was Julie’s point. Instead of expecting to implement DDD right off the bat, why don’t we approach it piecemeal?

Julie suggested, try to build it like a garden. Let’s start with a single piece, get it right, feel good about it and gain that confidence to continue. Start small.

Don’t tell. Do!
– Julie Lerman

Eventually, you may have this nice garden of DDD but start with a single pot.

Cherry Laithang

Practically speaking, she reiterated something I picked up in Uncle Bob’s Clean Architecture as well, boundaries. It’s all about how we separate things.

Designing Towards Event Sourcing

By Tugberk Ugurlu (video)

Mentions three good use cases/benefits for considering event sourcing:

Very well suited for audit logs
Help in troubleshooting issues since all history of data is stored
Allows for simple evolution in changes of what data needs to be stored

There are difficulties as well though, such as reading large aggregates dependent on multiple streams. As for other concerns such as GDPR, concurrency, eventual consistency, ordering, but, there are widely adopted solutions.

Strategic Design – The Joy of Multiple Models

By Henning Schwentner (video)

Just because in real life, a concept or object occurs several times in the same business, it does not mean you have to model it all in one piece.

Just like world maps come with different views, maps of rivers, maps of borders or height maps. Modelling the reality you will work with as you need to work with it reduces the cognitive load. Simply, bounded contexts. When purchasing a car, the process for completing the purchase might be completely different from the details of how you finance your purchase. Split the two apart and you will get a model for each which makes sense and can be grasped within its context.

Something else that many speakers have iterated which deserves mentioning again. Distributing to microservices isn’t the point, the point is to get separated pieces of functionality that help us understand and develop them separately. He had this great graph which illustrates this:

This aims to visualise how, if you have a big interconnected mess, it’ll just be a distributed interconnected mess if you split it up without properly building boundaries. Whilst at the same time, services that could be distributed might also live inside the same process as kind of a “modulith”.

Micro Frontends – A Strive for Fully Verticalized Systems

By David Leitner (video)

David led by saying that, in most cases, you won’t need this. In 80% of cases, using one big monolith frontend is probably fine.

It’s a recurring theme. I think I’m not alone in being quick to jump on the bandwagon as soon as I hear a new buzzword “micro frontends, cool, let’s do that!”. So thank you, David, for that disclaimer 🙂

That being said, if you have a site in which different teams do have, or maybe should have, full ownership over a part of the UI, you may want to give them that control. Designing for resiliency is another good use case. The popular example, which was also used by Richard Roger in an earlier talk, is Amazon. They use a mix of patterns to split the interface into parts, likely developed and deployed independently by different teams. The techniques that were showcased which might be worthwhile reading up on were:

Hypermedia splitting; simply linking between parts of the user interface
Build time includes; simply link in the parts of the UI in the build step
Server-side and Edge-side includes; using nginx or IIS/Apache to statically include parts of html
Server-side side application includes; have a shell application request parts of the UI from microservices when the page is requested
Client-side includes; from within a SPA application, request parts of the UI directly from microservices

All solutions come with different pro’s and con’s and should be used or better yet, mixed and matched, depending on the context.

Workshop: Show me the Kubernetes

Held by Bruno Boucard and Thomas Pierrain

Great down to earth workshop of how to very simply get a Kubernetes cluster up and running with Ingress and automatic failover. Check out the exercises: https://github.com/CloudNativeWales/ShowMeTheKubernetes/

They also highlighted a very interesting resource, the Cloud Native Foundation Landscape which showcases most if not all of the products out there in the clouds. It’s a really interesting view of just how much is going on and might give a good overview of things to investigate if considering a move to the cloud.

Keynote: Making a Case for Conceptual Integrity

By Diana Montalion (video)

So much good things in this talk, hard to pick the absolute highlights.

We all get emotional, angry even, in the face of issues we know of better solutions for. Or maybe we don’t, we just feel something is wrong. It bugs us and we get frustrated. But, screaming about won’t change anything, will it?

I think when Diana talked about ‘conceptual integrity’ she meant that it’s easy to see when it’s lacking. Stuff just doesn’t make sense. And instead of talking about in what ways or how we can shout about it she took a few steps back. She suggested that since our systems reflect our organisation what we need to think about is the leadership that brings about conceptual integrity.

Her most important point was that everything requires reasons. Problems are not debates, they are processes towards a solution, everything has reasons. Why is solution X best given the current circumstances? Facts, inferences and their weight.

Is it so, and does that follow and does it matter?

I especially liked her idea of integration leadership.

It’s about synthesising wisdom into sound decisions. Where wisdom = knowledge + experience + good judgement

And it’s important to note that it does not need to be your wisdom.

Summary of day three

Maps and diagrams are key. I always travel with whiteboard pens and post-its in my backpack and if someone in my team has a question, they’ll seldom get an answer until I’ve grabbed a hold of these and made my way to the whiteboard. I even keep a small A4 size whiteboard by my desk. What Simon Wardley presented as well as Mathias with the temporal modelling just strengthens my belief that visual representations truly are key in communicating. And honestly, isn’t much of what we do communication? As humans, we are great at remembering symbols and finding patterns in visual things so let’s use that. Whiteboards and models for all!

Keynote: Crossing the River by Feeling the Stones

By Simon Wardley (video)

Simon’s presentation was high paced and packed full of useful information. It’s a lot to summarise but so I’ll make only a quick list of the best takeaways and refer to further reading

By looking only at budgets or SWOT diagrams companies often miss the big picture. Lots and lots of money are spent on optimising bottlenecks instead of evolving the flow. What’s this mean? In his example, imagine a company that wants to digitalize their paper handling. They want to move over to digital scales for measuring incoming papers. Using one of his `Wardley Maps` he shows how instead of doing that, they could simply get the source data digitally instead of measuring papers. Probably a simplified case but still it shows very much how we miss the big picture and how maps can help us lift our eyes to a birds view perspective.

Wardley’s own medium page, wardleymaps
Sun Tzu’s five factors & John Boyd’s OODA loop; how we can approach challenges
GCHQ: Boiling frogs
https://strategy-madlibs.herokuapp.com/ a hilarious example of how company strategies can be generated based on a simple template

How To Build a Social Network Entirely on Serverless

By Yan Cui (video)

An interesting in-depth walkthrough of how their company moved from a monolithic application to serverless. I found these two concepts the most enlightening:

He showed the architecture before and after the move, and the monolith architecture looked simpler. So why move? Well, as he explained it, partially, they had easily added on functionality during the move which added more boxes. But also, and more importantly, it didn’t make the solution more complex, it only raised and highlighted the complexity that was already there as lambda functions.
Breaking the system into lambda functions truly show how easily extendable the system becomes (at least on the back end). They could very easily add on functionality, or move functionality, and then simply point their APIs and/or frontend to use the new services.

I think a lot of the benefits they saw much boil down to a well developed CI/CD framework and I believe this is key in how we develop software going forwards. The faster we can deploy something, the smaller we can make it.

But What about the UI?

By Thomas Presthus (video)

This talk followed along the same lines as the Micro Frontends one by David Leitner and I apologise if I get some of the content mixed up. Both were very good and Thomas had some in-detail explanation of how things can be set up. Look at these two resources from his talk:

The Documents to Applications Continuum on how what your website does may affect which patterns you want to use
https://micro-frontends.org/

Beyond Design Principles and Patterns: Writing Good Object-Oriented Code

By Matthias Noback (video)

There were lots of practical down-to-earth examples of how to write good code, all explained without complex terminology but what I believe we can really take from this talk is that principles and patterns don’t matter nearly as much as the very basics of how to write objects. Stick to the basics and that’s often good enough to just make your application click.

introduce more types; if the input is an email, use an email, it’s not a string
be more strict about your objects; show more behaviour and less state
design objects that only accept valid data; don’t let the caller add -1 items to the cart
design objects that are always valid; if your object has an isEnabled false, surely, there’s something that is not allowed after it has been disabled
use composition instead of inheritance; need to change your class? break out the functionality and inject it in the constructor

Lightning talk: Independent Services, Throw Your Hands Up At Me

By Heather Whyte (video)

Not all microservices need to be 100% independent. But we want to keep them as independent as possible. Why? Because otherwise, we lose the benefits we wanted from having microservices to begin with. Is there an easy way to gauge this? Yes, three simple questions:

Can I easily turn it on?
- Do you require things at the build stage?
- Does building require other things to rebuild?
Can I safely turn it off?
- Does it take anything else down with it when it goes down?
Does it own its own data?
- Do we have full control of the data or will something else break our microservice when changing?
- Will changes to our microservice break something else?

Keynote: Temporal modelling

By Mathias Verraes (video)

What Mathias raised in his keynote was how the UML kind of view simply does not model the behaviour of the system. And really, we build systems for behaviour, not data.

What he suggests instead is a different approach to modelling, one based on events, what happens in the system. This allows us to look at how we store this in different ways. Different services may interpret the events very differently. Think for example of a dashboard, it would want to count all cancelled accounts, if we simply store a list of the accounts in the system, well, that kind of data is impossible to infer. So, it’s time for a new type of modelling, one based on what happens in the system and through that, we can then think of how we need to infer our data.

Looking at the dependencies between events and how they naturally fall together is also an excellent way to find our bounded contexts. Things naturally follow the same language in this kind of view. It also helps track down temporal bugs that could be virtually impossible to see otherwise.

In conclusion, it all depends

The going joke at the conference was “it depends”. Sometimes referred to as the contractor’s or architect’s go-to phrase. And it does, doesn’t it. It all depends on the context. Once we figure out exactly what the context is, time, money, resources, knowledge, then we can make our decisions based on our best knowledge. The conference definitely had a strong pro-pragmatism.

Unfortunately, I missed the workshop on event storming. It was referred to by many and seeing how the maps presented by Wardley and temporal modelling by Verraes made such sense to me, I believe this is the next good thing to add to the portfolio.

If you made it this far in this post, truly, you are a champ! I thank you for reading and I hope you found something worthwhile from all the random tidbits of knowledge I managed to plot down. And if you disagree, was there and I got something wrong or have something to add, I’d love to hear it and I can amend as needed.

Thank you!