The More I Know, the Less I Know

I used to be very confident in my abilities as a software developer.

I used to be able to walk up to a group of software developers and tell them exactly what they were doing wrong and exactly what was the “right” way to do things.

I used to be sure of this myself.

confident thumb The More I Know, the Less I Know

It wasn’t even that long ago.  Heck, when I look at the blog posts I wrote 3 years ago I have to slap myself upside my head in realization of just how stupid I was.

Not only was my writing bad, but some of my thoughts seem so immature and uneducated that it feels like a completely different person wrote them.

And I wrote those posts back when I knew it all.

The more I learn, the less I know

Lately I’ve been running into situations more and more often where I don’t have a good answer for problems.

I’ve found myself much more often giving 3 pieces of advice attached with pros and cons rather than giving a single absolute—like I would have done perhaps 3 years ago.

I’ve been finding as I have been learning more and more (the past 3 years have been an accelerated pace of growth for me,) that I am becoming less and less sure of what I know and more and more convinced that there is no such thing as a set of best practices.

I’ve even spent some time postulating on whether or not commonly held beliefs of best practices would be thrown completely out the window given a significant enough motivation to succeed.

My point is that the more doors I open, the more aware I become of the multitude of doors that exist.

doors thumb The More I Know, the Less I Know

It is not just the realization of what I don’t know, but also the realization of weakness of the foundation I am already standing on.

Taking it out of the meta-physical

Let’s drop down out of the philosophical discussion for a bit and talk about a real example.

Perhaps the biggest quandary I struggle with is whether or not to unit test or practice TDD and its variants.

The 3 years ago know-it-all version of me would tell you emphatically “yes, it is a best practice and you should definitely do it all the time.”

The more pragmatic version of me today says, in a much more uncertain tone, “perhaps.”

I don’t want to delve into the topic in this post since I am sure I could write volumes on my ponderings in this area, but I’ve come to a conclusion that it makes sense to write unit tests for code that has few or no dependencies and that it does not make sense to do so for other code.

From that I’ve also derived that I should strive to write code that separates algorithms from coordinators.

I still even feel today that my advice is not wholly sound.  I am convinced it is a better approach than 100% TDD and units tests, or no TDD and unit tests, but I am not convinced there isn’t a deeper understanding and truth that supersedes my current thoughts on the matter.

As you can imagine this is quite frustrating and unsettling.

Silver bullets and best practices

What I am coming to realize more and more is that there are no silver bullets and more surprisingly there are not even any such things as best practices.

silverbullet thumb The More I Know, the Less I Know

Now I’ve heard the adage of there being no silver bullets so many times that it makes me physically sick when I hear someone say it, because it is so cliché.

But, I’ve had a hard time swallowing the no best practices pill.

I feel like when I abandon this ship then I am sailing on my life raft in the middle of a vast ocean with no sails and no sense of what direction to go.

A corner-stone of my development career has been in the learning, applying and teaching of best practices.  If these things don’t exist, have I just been pedaling snake oil and drinking it myself?

No.

Best practices are simply concrete applications of abstract principles in software development that we cannot directly grasp or see clearly enough to absolutely identify.

Breaking this down a bit, what I am saying is that best practices are not the things themselves to seek, but through the pursuit of best practices we can arrive at a better understanding of the principles that actually are unchanging and absolute.

Best practices are optimal strategies for dealing with the problems of software development based on a particular context.  That context is primarily defined by:

  • Language and technology choice
  • Meta-game (what other software developers and perceived best practices are generally in place and how software development is viewed and understood at a given time.)
  • Individual skill and growth (what keeps me straight might slow you down; depends on where you are in your journey.)

There is a gentle balance between process and pragmatism.

When you decide to make your cuts without the cutting guide, it can make you go faster, but only if you know exactly what you are doing.

Where I am now

Every time I open my mouth I feel like I am spewing a bunch of bull crap.

I don’t trust half of what I say, because I know so much of it is wrong.

Yet I have perhaps 10 times more knowledge and quite a bit more experience in regards to software development than I did just 3 years ago.

So what gives?

Overall, I think I am giving better advice based on more practical experience and knowledge, it is just that I am far more aware of my own short-comings and how stupid I am even today.

I have the curse and blessing of knowing that only half of what I am saying has any merit and the other half is utter crap.

Much of this stems from the realization that there are no absolute right ways to do things and best answers for many of the problems of software development.

I used to be under the impression that someone out there had the answer to the question of what is the right way to develop software.

clues thumb The More I Know, the Less I Know

I used to think that I was picking up bit of knowledge, clues, that were unraveling the mystery of software development.  That someday I would have all the pieces of understanding and tell others exactly how they should be developing software.

What I found instead was that not only does nobody know the “right” way to develop software, but that it is perhaps an unknowable truth.

The best we can do is try to learn from obvious mistakes we have made before, start with a process that has had some level of success, and modify what we do based on our observations.

We can’t even accurately measure anything about software development and to think we can is just foolishness.

From story points, to velocity, to lines of code per defect and so on and so forth, all of those things are not only impossible to accurately measure, but they don’t really tell us if we are doing better or not.

So, what is my point?

My point is simple.

I have learned that not only do I not have all the answers, but I never will.

What I have learned is always subject for debate and is very rarely absolute, so I should have strong convictions, but hold onto them loosely.

And most importantly, don’t be deceived into thinking there is a right way to develop software that can be known.  You can improve the way you develop software and your software development skills, but it will never be based on an absolute set of rules that come down from some magical process or technology.

If you like this post don’t forget to or subscribe to my RSS feed.

  • http://twitter.com/geoffmazeroff Geoff Mazeroff (@geoffmazeroff)

    I think the Dunning-Kruger effect strikes us all at some point in our professional lives. I really enjoyed your comments about how you’re dealing with it. This article is useful to developers at any stage in their career. Thanks for the post!

  • Henry Buckley (@henrydbuckley)

    Kudos for voicing your internal questioning so honestly and eloquently. It’s a sentiment I’m sure is shared by more than are willing to publicly admit it, and for my money much better to continually question and embrace doubt in our industry than to adopt the opposite evangelistic mindset.

    @geoffmazeroff Doesn’t Dunning-Kruger relate to overconfidence in the incompetent – i.e. the very opposite of what is conveyed by the post. ‘Imposter syndrome’ would seem to be a closer match.

  • http://jhonatantirado.wordpress.com/ nathan

    Reblogged this on Nathan.

  • http://dangerismymiddlename.com Paul Danger Kile

    Unit testing? Yes. TDD? Yes: both.

    But that’s not really what you were asking. :-)

    I am most proud of the simple solutions that take the place of large systems.

    The best tools that I have for finding these are:
    – To force myself to only consider business goals. The first step to that is to practice defining Essential Use Cases first.
    – Use tech at home that I don’t use at work. That way a Java developer might find a solution that uses a spreadsheet as DBMS frontend, instead of spending weeks on development.

    Of course, in the end, we build what they pay us to build, but there will be opportunities to really save time, effort, and money
    , at some point, and we will be more ready for them.

    Thank you for writing.

  • http://tipsdegabo.wordpress.com/ Gabriel Espinoza Erices

    jsonmez,

    I have about 3 short years of experience in software development and let me tell you that I’m starting to see what is happening to you. The more I think I know, the more I have to search and learn to give ‘accurate’ answers, so I rather taking a choice and follow that choice blind-folded instead of wasting my life seeking answers I will never find.

    BTW, you’re making an excelent job with your blog, that is certain!.

  • http://twitter.com/ntcoding Nick (@ntcoding)

    I start most sentences with “in some context this might be a good idea…”.

    It’s funny how we both feel we are “talking bull”.

    My approach is to think about things and be more analytical. It’s the reason I created this tool http://www.mylifelessons.net/ to help me remember all the important things I learn and ways of think about certain situations.

    It feels great doesn’t it when you stop seeing SOLID principles (and all the others) as a goal but merely a guideline?

    Now your next problem is that all the people you work with haven’t yet realised this. And because you don’t follow the “best practices” al the time you’re not that good a developer.

  • Pingback: EA Voices | Best practices – adapt, then adopt()

  • http://tamasrev.wordpress.com tamasrev

    Reblogged this on tamasrev.

  • http://tamasrev.wordpress.com tamasrev

    I believe, doubts are inconvenient, but at least, they make us think.

  • kamran

    Enjoyed it!

  • http://fezilenkuna.wordpress.com Fezile

    I wanna start programming

  • Pingback: Programmer Competency Matrix – Education and Experience | WORD PTR - Pointed Development()

  • http://slapotics.wordpress.com slapout

    Reading your post leads me to start thinking that maybe “Best practices” comes from this: weighing the pros and cons of each technique. And then the technique that has the most pros and least cons in most situations becomes the “best practice”.

  • EECOLOR

    A few years ago I realized a similar thing and got I back to thinking about what I liked. I eventually found out that I love to work with code, treat it like clay and work on it as if it’s an arts project.

    Back then I went looking for a language that I wanted to use. I had a few wishes, which I realized were completely subjective. Anyone that tells you language X is better than language Y might think twice. Other people that say you need the right tool for the job probably have a good idea of what they think is the right tool for that particular job. Eventually I found Haxe and a short while after that I found Scala. That sais little about about Scala and all about me and my opinions of what I would like to see in a language.

    It has been a good choice for me as Scala let’s me rewrite my code over and over again. I first got introduced to compiler plugins. After a while I found out about functional programming. I met macro’s and then somewhat later I found out about type level programming. All in all I made a lovely choice with a language that allows me to think about how I want to express things. It let’s me tinker with my code over and over, continually improving it. Note that I am not advocating Scala, I just found the clay I love to work with.

    As for work, I just do it the way that feels right at the moment. When I look back at code I previously wrote (sometimes only a week ago) and smile, it’s like traveling back in time.

    It feels like I am part of a guild and study hard to improve my craft. For me the end result is not that important, spending time and effort on my craft is.

  • Pingback: Principles Are Timeless Best Practices Are Fads « Making the Complex Simple()

  • Pingback: Новости » Blog Archive » [recovery mode] Принципы — вечны, практики — иллюзорны()

  • numi numful

    The fact that you came to this realization shows who you are.
    This realization is far from being trivial, but is an impotant milestone in one’s journey

    The great Socrates said it perfect 2500 yrs ago:
    “As for me, all I know is that I know nothing, for when I don’t know what justice is, I’ll hardly know whether it is a kind of virtue or not, or whether a person who has it is happy or unhappy.
    Republic, 354c, (conclusion of book I)

    • jsonmez

      Thanks, that is a good quote.