dateo. Coding Blog

There are patterns in our day-to-day life, in our relationships, in our jobs, that are worthwhile to be aware of. For example, if I were a person to constantly overreact to other people's opinions, it would be good to be aware of this, so that I could try to own it, maybe even change my behaviour around those kinds of triggers.

One of the patterns that are very obvious in the community of software engineers and developers is the so called Dunning-Kruger effect, which you may have heard of already. If not, let me introduce you.

The Dunning-Kruger effect

Let's first have a look at the definition Wikipedia is giving us:

The Dunning–Kruger effect is a cognitive bias whereby people with low ability, expertise, or experience regarding a certain type of task or area of knowledge tend to overestimate their ability or knowledge. Some researchers also include in their definition the opposite effect for high performers: their tendency to underestimate their skills.

So, we are dealing with a cognitive bias or, in other words, a misconception about our own capabilities. The symptoms of this effect are said to be found in various fields such as politics, driving, medicine - it is not something that directly relates to the software industry but rather to all areas, where skills are required to do something, and where people estimating their own skill is somehow involved.

The different stages

The Dunning-Kruger effect typically unfolds through different stages for which the independent variable is the "wisdom" or knowledge around a certain topic. To illustrate this, I am borrowing a figure by Ulf Ehlert, 2015

As you can see, as someone, a person, gains knowledge or wisdom around a certain topic or field of expertise, their confidence undergoes different stages with highs and lows.

Know-nothing

The earliest stage is the start of any learning process, where you actually know that you don't know anything. I am sure you felt like this at some point in your career, whether it being when starting a completely new programming language, a new framework, in a new team, or anything similar. You don't know anything and you are also not very confident in accomplishing anything productive yet. So you set about learning and practising as much as possible.

Mt. Stupid

Some (short) amount of time has passed, you read many tutorials and debugged all hello-world samples, found answers to the most trivial questions on stack overflow, you even built a button into that sample that changes its colour when hovering over it on your own, or maybe you outer-joined the person and the address tables using raw SQL. So that's it, right? Conquering the world has become possible. What problem could possibly be out there, that you were not able to solve with your newly acquired skill set?

(See yourself in there somewhere already? I certainly do.)

Valley of Despair

The descent from Mt. Stupid can be initiated by various triggers. Could be you are assigned a real-world problem in that domain, or maybe someone asks you a question for which in-depth knowledge is required. Or maybe you simple continue to learn and about the thing you started with.
In any case, a sudden (or sometimes not so sudden) realization kicks in: Actually, you still don't know anything profound yet.

There is no way you could satisfy a customer with what you know so far, no way to actually develop something useful and make money from it, no way you would survive an in-depth code review with one of your senior developers.

But the good news is: You are now aware of this. And you can make your situation even better by publicly admitting that you still have much to learn here.
Personally, I find that the best people to work with are those that are open to admit that they don't know everything.

Future

Every despair grows hope - and if you dare to pursue, the only way forward is upward again. So you continue to learn, experiment, discuss with peers, ask questions and slowly regain the confidence you had when you actually didn't know much - except this time, you really do know something already. Keep going, and you are on a path to become an expert in something.

Ill-placed Complexity in Software

I am pretty sure that if you've read this far, you could in part either relate to the things I spoke about, or at least think about some situation or even a colleague that would fit into this schema.

I want to extrapolate the effects of Dunning-Kruger one step further to the domain of software engineering. To me, it is not a coincidence that many memes and comics around software development feature a similar overall shape to the diagram above. Take, for example, this figure:

Again, you might have seen this before and you'll probably have smiled at the inherent truth around it, but, actually, it is sadly true.

This graph depicts the more general progress of us software developers during our careers and it is highly correlated to the implications of the Dunning-Kruger effect.

Know-nothing

You start out as a newbie in your field, barely managing to get even the simplest things done. People assign you only with the tasks they think you're capable of accomplishing - your code is simple, because you don't know how to do otherwise.

Mt. Stupid

You are learning from your peers, you took courses, read books and memorized all 273.5 design patterns by heart. You are ready to conquer the world with your skills. Also, your code is starting to get more and more complex. "That's how the experts do it!", one might here you say. Or, "But we might need that later." Maybe even "But wouldn't it be cool if...", which is one of the gravest mistakes we can make according to Billy Hollis.

In fact, your code a gained a level of abstraction and complexity that, yes, it might be totally generic, but everyone else needs a few days until he understands it, your colleagues are afraid to make changes because it causes compilation errors in dozens of places and it is abstract to such an extent that no one knows the actual implementation until the start the application in debugging mode.

Damn it. You are such an expert.

Recovery

We need to make a small detour from the phases of Dunning-Kruger. Usually, in case of the pattern that I just described, there is no single thing to trigger the descent into the valley of despair. Instead, people slowly start to realize, that their code is too complex, too hard to understand. How? Well, experience, for one. But hopefully there are more senior engineers around and people willing to speak up and demanding code that everyone could maintain.

If this criticism is well-meant and positive, it will spark a slow progress of realization. One cannot simply start writing simpler code. That is a craft in itself to be practised and honed and it'll take time.

In the end, the good programmer will write simple code again. This code is not simple because of lack of knowledge though. It is feature-rich, based on design patterns, but it is easy to understand and work with. And is something that we all should aim for in the long term.

Roundup

To be honest, I think that the majority of us go through phases of the Dunning-Kruger effect regularly. And if you are, like me, a software engineer, I claim that you are going through the stages of ill-placed complexity as well. And also: That's totally fine. Sometimes, there is just no way to avoid it, we are human after all (not true for ChatGPT which is a KitchenAid taking human nonsense as input, stuck on Mt. Stupid).

I am only advocating for more awareness around the things we don't know. If you are willing to question your own decisions, guidelines, your own code, then you are on the right path to becoming a good developer.

Please share on social media, stay in touch via the contact form, and subscribe to our post newsletter!