Why focused domain models are better than generic ones

In software we like things to be generic and reusable. When designing data models we try to predict the future and possible scenarios where system can evolve. We often forgot that to be reusable, first system must be usable.

I like generic mechanisms. Generic search engine, generic RPC library, generic audit, generic authorization framework, generic UI components. Reusability on technical level is great. You can quickly build new things from existing blocks.

But generic business model? It is much easier to work with business domain model when it is focused and very specific. There should be no properties, classes, database tables or relationships just in case for the future. Every domain class and field must have specific, well defined meaning and purpose. There should be a real-life story behind each domain element. There should be no place for misinterpretation. Source code is often the most reliable source of truth about how whole organization works. Source code and its data must not lie about what it can really do.

I like to design systems by starting with business use-cases, end-to-end user flows and data flows. Once you have those dynamic elements of the system defined, then data model to support the processes should be straightforward. It is the business process to define domain model, not the other way around.

Development teams tend to focus too much on generalization to support unknown processes instead of pushing stakeholders to define concrete processes. Let’s do not pretend that we are prepared for all possible future scenarios. Both in business and in code. Let’s focus on bundling something that works and is well defined. This is why we adopt agile methodologies – to be able to evolve our models in iterations. This is why we adopt microservices – to be able to retire outdated process easily and build new ideas independently.

To conclude. Avoiding generalization on business domain level is good. It is not an anti-pattern. Generalization is often a distraction from the real goals. Generalization makes the goals less achievable and vague. Let’s be focused and specific.

Learning from work experience vs self-studying

I’ve share in one of my articles that is is estimated that only 10% of the learning happens a the formal training. The remaining 90% comes from everyday-tasks and learning from coworkers. Formal training includes things like self-studying from online resources which I would like to emphasis in this article. Since it’s only 10%, can it be treated with low priority?

Self study – 10% of the time

I do not know how those numbers were calculated. I’ve seen those numbers in on of the managers training. In my opinion those numbers can be quite accurate when we think about the time that we are able to spend on learning. Most of the time we spend at work and this is where we have the biggest opportunity to learn. Work builds real experience and practice. The knowledge becomes not just theoretical but also tested in real-life. We are able to come up with our own use-cases, examples and experience practical challenges.

Studying vs practicing vs teaching

When we think about the levels of knowledge it may be illustrated as: student → practitioner → teacher. Self-study brings you only to a student level. Good courses include hands-on labs, so that you can experience also some practice. But training exercises are always simplified and cover only simple “happy paths”. They do not include production-level challenges. It’s like fight with a shadow vs fight with a real opponent in martial arts.

Self-study – the impact

But does it mean that self-study can be ignored as it contributes only 10% to your learning? Absolutely not. It’s 10% in terms of the time but can be much more in terms of the impact. We do not always have the comfort to learn new things at work. Especially when you are an architect or in general technical lead, your company expects that you are the one who teaches others, who knows the new trends and who is up to date with latest technologies. You have to do a lot of self-study so that the whole company does not settle down with old solutions.

Online resources for self-studying

I was recently writing about DNA program which is a great example of self studying materials for software architects. Recently I’ve also singed-up for Google Cloud Platform Architecture training with Coursera. After doing the first module and getting this certificate I can definitely recommend Coursera GCP trainings. The best thing is that the training include a lot of hands-on labs with real GCP resources provisioned via Qwiklabs platform.

I have to admit that the online training possibilities that we have now are amazing. For a small price you can have access to resources that are often of much better quality than (unfortunately) some lectures at stationary universities.

Summary

So, go and self-study! Then use it at work and build a better world 🙂

Software developer 2020 vs 2010

I was starting my career in software development in 2008. We are now in 2020. What seemed to be true in 2010 and seems funny in 2020?

  • jQuery is the best thing that could happen for JavaScript and best library ever
  • Internet is not usable on phone. You need a desktop browser
  • You need to be a Linux or Windows guru to setup a production-ready server
  • SQL database is the most important component of the system. You cannot just store data in files
  • We do not have any unit tests and are not worried at all
  • Agile methodologies is a new thing. We have read about it and maybe will try in a new project after the specs are ready
  • Big data is a new thing and it’s about creating OLTP cubes
  • Machine learning is a niche academical subject, not usable in practice
  • Functional programming is a niche, good only for mathematicians
  • XML is nice, you can use cool XSLT transformations with it
  • You cannot be hired as a programmer after 3-weeks course
  • You can use cookies and users do not have to know

In 2010 you could say those things and be promoted to a team leader as a very wise person.  How does it sound in 2020? 🙂

To get even more sentimental you can also check this article as a nice summary of last decade in software development: https://blog.pragmaticengineer.com/the-decade-in-review-in-software-development/

Monorepo in Azure DevOps

When working with projects using microservices architecture I opt for monorepo pattern. A single repository containing source code of all the services. It facilitates knowledge sharing between teams and encourages more unified programming style. Microservices give a lot of technological freedom, but this freedom should be used wisely. Common standards across the organization are still important.

Working with a single repository forces all the programmers to at least have a glance at the list of commits from other teams when making git pull. This channel of knowledge sharing can be very beneficial.

Despite having a single repository, we still need separate pipelines and policies for separate directories. Azure DevOps facilitates it by allowing directory path filter in crucial places:

  • path filter in build trigger settings
  • branch policies
  • required reviewers

It allows to setup a clear ownership of different parts of the repository and apply different pipelines to different parts of the repo while still having all the benefits of monorepo pattern.

Kubernetes basics for Docker users

The aim of this article is to build a high-level mind-map and understanding of concepts like Kubernetes, Helm and cloud-native applications. The assumption is that you have worked already with Docker.

Docker

So you know Docker. Docker container is like a virtual machine but lighter. Why lighter? It does not have operating system inside. It relies on host operating system. Docker adds only applications layer on top. You can run many Docker containers in a single server / virtual machine.

Docker Compose

You may have also heard about Docker Compose. For example when your application consists of .NET Core web server, MySQL database, and Redis cache – you can define 3 separate containers for it. To run all of them in a virtual network you define a docker-compose.yml file. Then 3 of them can be run with a single docker-compose up command.

Scaling the application for production

Now let’s imagine you want to scale your application. You introduce a load balancer and 2 additional web server containers. You are also adding a RabbitMq container and an instance of a background processing worker. There are also other requirements for production environment:

  •  containers need to be distributed across many servers
  • containers which do not need much resources can be run together in same server to use provisioned servers in a cost-effective way
  • when a container is not responding it should be restarted
  • when connectivity with container is lost it should be replaced with a new instance
  • number of containers should autoscale
  • number of servers should autoscale
  • new containers added to this environment should be auto-discovered
  • it should be possible to mount and share storage volumes in a flexible way

Things are getting complicated. To achieve all that requirements we must write a lot of code to monitor and manage infrastructure. This is called containers orchestration. Or… we can use Kubernetes (k8s) which has all that features and more built-in!

Kubernetes concepts

Pod

Abstraction of a single app. It can have one ore more containers. If containers are tightly coupled they may be placed in same pod. All containers inside a pod share storage volumes. A pod is an unit of deployment and scalability. Each pod has IP address assigned, so there is no need to care about port conflicts.

Node

This is how k8s names physical servers or virtual machines hosting the containers.

Cluster

Set of nodes available for k8s. Example cluster would be 4 nodes and 20 pods running on them, managed dynamically by k8s.

Namespace

All objects within a cluster can have a namespace. It allows to create many virtual clusters inside single cluster. It is useful for example to model many independent environments for staging in a single k8s environment.

Service

Since each pod has it’s own IP and pods can be started and shut down, it would be not easy for other pods to keep track of constantly changing IPs. That’s why we have Services in k8s. Service groups a set of pods by given labels. Pods may come and go, but as long as labels criteria are matching, all matching pods are automatically tracked. Service has a logical name assigned, so that other pods can use this name to communicate with the pods behind the service. Service routes and load-balances the traffic dynamically to relevant pods.

Services can be also used to point traffic to an endpoint outside k8s cluster. In this case instead of defining a service by providing pod labels selector, it is necessary to define an IP of the service backend.

Ingress

Ingress is used to expose services to outside word via http(s). It can also terminate SSL.

Deployment

Deployment specifies the pod and the number of its replicas that should be run. Deployments controller is responsible for rolling out updated pods (e.g. with updated container image). It starts new pods, shuts down old pods and then keeps monitoring them to make sure that desired number of replicas is run.

StatefulSet

StatefulSets are used to manage containers which contain data. Containers that have data cannot be just removed and replaced as we cannot loose its data. Pods in StatefulSets have sticky identities and persistence storage assigned. Persistent storage is not deleted when pod is deleted.

Worth to mention that in many scenarios managing persistence would be simpler outside Kubernetes cluster. Many cloud providers have sql an noSql as-a-service offerings which usually takes care about things like backups, availability and replication.

Monitoring containers

Each container has a liveness and readiness probe defined. Typically those are HTTP endpoints called by Kubernetes to check if container is healthy. K8s calls the endpoints periodically, e.g. every 10 seconds depending on configuration. When liveness probe fails container is restarted. When readiness probe fails traffic is not anymore routed to this instance. Health check endpoints must be implemented in every service. Simple liveness endpoint could be just returning status code 200. Readiness endpoint could additionally check things like database connection, cache readiness or amount of currently used resources to check if service is really ready to process new requests. When readiness endpoint detects a problem that could be solved by restarting the container, it could potentially switch a variable to force liveness endpoint to fail causing a restart.

Helm

An application targeting Kubernetes is configured in a set of yaml files for deployments, services etc. Those sets of yaml files can grow pretty complex. We also need some versioning tool for them. A common approach is to package all k8s files into a Helm package called a chart. What is being deployed to k8s cluster is a chart.

Cloud native applications

Kubernetes is an operating system for cloud native applications. Cloud native applications are usually designed in microservices architecture and aim to be cloud-agnostic – can run in many public clouds or in a private/hybrid cloud. There is already a lot of predefined Helm charts available in public repositories if you’d like to pull scalable k8s setup e.g. for Cassandra or Prometheus in your cloud-native setup.

Sources

What matters in creating software?

What are the most important things when building software? Is it architecture? Frameworks? Process? No. Those are only tools.

What really matters is the fact of delivering useful solution. Usefulness defined in terms of number of users or revenue. Useful solutions are not delivered too late or too early – they are delivered on time. Useful solutions are not too slow or too buggy – they have acceptable performance and are reliable.

What also matters is the satisfaction of the team after building the solution. It could be that kind of satisfaction which painter has when working on a painting – satisfaction during the process. There can be also some pain in the process but that would be another type of satisfaction – the one that athlete has after finishing a marathon. It was painful but feels so good after the finish. It could be also like a satisfaction of a cook who has just discovered new spices and learned something useful for the future.

Studying architecture or frameworks is also important, because it makes us efficient and more likely to deliver useful solutions. But the goal is not to learn, the goal is to apply the knowledge and make things happen.

Employee development plan

I was recently participating in managers workshop at my current company Mash. One of the learnings was about employee development plans. Please find below the key points that in my opinion should be always considered when crating development plans.

Not everyone needs a development plan

Many employees may feel good at they current position. It is not always appropriate to push for more. People may want to specialize at their current role and may be doing a great job in this role. Managers should not be pushing for “next” levels. Especially if it requires to change focus to other things. Not everyone who is a great engineer would be a great team leader or great designer. Adding new responsibilities may affect ability to continue the great work that given person is currently doing.

There are moments in career that people may actually want to change something and this is a proper moment to have a development plan which would support employee in reaching next personal goals. But it most cases it is far more important for employee to be able to fully engage in the current role. Development plans should be not a replacement for that.

Where the development happens?

If everything is working well in our workplace, here is where learning happens:

  • 70% of the learning comes from everyday tasks
  • 20% from colleagues (peers or mentors/leaders)
  • 10% from external sources like formal training, conferences or books – that are  great ways to get some initial knowledge but what makes an expert is practice and solving real problems, not reading articles.

This division should be also considered when designing development plan. Employee may spend many days at conferences but if everyday work is not giving opportunities to develop and use that knowledge – then development activities are far from being effective.

Who needs a development plan?

When preparing development plans, first think about poor performers. They may need extra attention to help them reaching expectations. Employees may perform below expectations because they did not have enough internal training and knowledge sharing to be able to carry on assigned tasks. If all the necessary knowledge sharing was done and there is still no improvement, then employee improvement plan should be discussed instead of development plan.

Also remember about your “rocks stars”. They usually also need extra attention to make sure that they have enough challenges to grow even bigger.

The most important is to listen to your team members to know what they really need. Development plans should be an outcome of real needs and expectations. As all other things in teamwork – it should not be based on assumptions.

DNA – week one

What is DNA?

DNA comes from Polish name “Droga Nowoczesnego Architekta” – it means: “The road of modern architect”. English abbreviation would be “TROMA”. This one of very little examples where Polish is simpler than English 😉

This is a 19-weeks course crated by 3 experienced architects who are also trainers. They are supported by popular blogger Maciej Aniserowicz who is the publisher of the course. He made the program well known thanks to the broad IT audience in Poland following his devstyle.pl blog and social media channels.

Course is dedicated to senior software developers and architects. The goal is to propagate modern patterns in software development. Presented theory is backed up by cases from real-life. It is meant to be the “killing feature” of the course. In addition to that there are also practical exercises after each week. The idea is  that participants get not only theory but also examples from real life and exercises to practice.

Visit droga.dev to learn more about DNA.

First impressions

I was following more ore less what was published by DNA mentors as “teasers” in recent weeks before full program became available. I was prepared to expect good content and I was not disappointed. The content is solid and also the way of presenting it is fully professional.

Good content was not a surprise for me after buying access to the course. I believed that those guys will do great stuff. I knew what I am buying. When you go to good restaurant you expect to get good food, no excitement here. But there was one thing that was a bonus that I was not expecting to be so meaningful: the community.

Joining DNA community on Slack was pure fresh excitement. This is a closed group for all course members. Joining this group was like joining tens of new teams in different companies in one day. People share and discuss how they approach various topics in their projects. This is great, tons of information and different points of view! You can read about an useful tool, online resource or interesting approach to specific problem. I expect that the value of content generated by the community may exceed the value of the course content itself as the time goes. Of course it’s Slack so no structure is present here and it will be impossible to find anything after couple of weeks 😉 But among all possible distractions you can have turned on, this Slack community is the beneficial one and can really broaden your mind

Eye opener #1

I like to search for analogies between constructions and software industry. In general there is a lot of analogy, but one of them seemed harmful to me: software developer is a creative role whereas constructions developer most often is doing a repetitive job based on detailed project. DNA explained that discrepancy – software developer should not be treated as construction worker but as constructor.

Developers write code which is a kind of design. In software the role of worker is taken by compiler and build pipeline. Having code written, we can then start many instances of running program created almost automatically. How beautiful it would be if we had “compilers” for building construction projects that would automatically execute design documents to create a real building?

Disclaimer to all construction workers who read that: if you don’t have specs from architect/designer/constructor, then you are also a creative worker creating something out of nothing 🙂

Looking forward for more eye openers when continuing with the course 🙂

Divide, eliminate and conquer

Ancient Romans were saying “divide and conquer”. I really like this rule both in system design and project management. To adjust it to contemporary times which are full of distractions I would also add “eliminate” phase. It helps to avoid conquering false targets and allows to focus only on biggest value goals.

Please find below some examples of what this rule means in practice.

Divide

  • Split complex problem into a number of smaller problems
  • Isolate application modules
  • Assign well defined responsibilities to teams and individuals
  • Break up project into phases
  • Define sub-tasks for stories to implement
  • Set clear SMART goals

Eliminate

  • Identify problems that can be ignored
  • Reduce scope
  • Remove distractions
  • Ask “why?”
  • Focus on goals, not procedures

Conquer

  • Execute goals with deep believe
  • Don’t give up
  • Improve
  • Know when is enough
  • Celebrate success

“Divide, eliminate and conquer” approach often helped me when I was feeling overwhelmed by a difficult problem or had to choose which path to go. I believe it is useful in wide range of situations starting from fulfilling New Year’s resolutions up to setting and executing enterprise strategy.

Alternative to traditional daily meeting

Most of the dev teams have daily meetings in the calendar. Daily meeting is a good opportunity to share with other team members what’s in our head currently. You may get help, recommendations or share your knowledge and  useful discoveries.

Daily meetings is also a good way to build the team spirit. Especially for distributed teams it is often the only chance during workday to see each others via video and hear each others voices.

But when following traditional meeting structure, we often can notice that team members are almost everyday saying something like: “Yesterday I was working, today I will be working, everything is OK”. Status is visible on board, others can see the contributions, why to waste time on going into details? Sometimes it’s indeed challenging and even uncomfortable for team members to say something more.

If this is your situation and dailies start to look as a waste of time and being uncomfortable for the team, your team may try a different approach: going through items in current sprint/board instead of going through people. You can find an interesting discussion about this approach here.

You may also mix both approaches by going by items for example on every 2nd day. The most important thing is continuous improvement and trying what works best for your team.