Thanks,
drhowarddrfine very interesting in deed.
This discussion is as old as programming languages themselves. And for sure I don't want to start it again.
As far as I can say, there is no silver bullet.
An infinite number of improvements are possible, but perfection cannot be reached.
One has to differ between if you're working alone, or in a team. And in a team you have to follow their guidelines - doesn't matter if you agree, if they are right, or wrong. Because starting a discussion on it while a project runs will only bring a lot of disturbance, lower productivity, and anyway will not bring up the silver bullet. (I excluded projects which guidelines really needed to be revised. But those often have other, more urgent issues than just their guidelines. [Don't ask! I will not name any. It's my very personal opinion. And I don't want to start any brawls.])
Programmers simply hate to document their own code. IMO many don't see the need for it, because they understand it themselves - when they just wrote it. And I know from myself: When the shit finally runs, the need for documenting it afterwards is just

Right? That's why I said in my former post to start with it, and do it parallel to the coding.

And while programmers are good at coding, writing prose, especially about code, is a complete other challenge, but also needed, and can be learned. And you can put only the form into guidelines, not the thinking.
But that's no reason for not doing it at all, nor for refuse to improve it.
While that's clear to me, I know there are many now want, and will contradict:
Writing documentation about the code has at least the same importance as coding itself.
"Would you rather spend 10 days reading 100,000 lines of code, or 4 days reading 1000?" is a very good point.
My (very personal) opinion is: The claim to prefer easy to read code over efficient code is attributable to the fact that it's almost impossible to bring programmers to f###ing document their f###ing code. So, when in any case there will be no useful documentation, or none at all, anyway - well, then the code has to be written as easy to read as possible.
There are high-flyers like Arthur Whitney, who can comprehend even weird code by just reading it and don't need no documentation at all.
But I dare say those ain't the majority.
And to refuse to document the code and accuse others just being too stupid to comprehend it without any documentation, does not make one a high-flyer!
To me:
When it improves efficiency I am pro weird looking code -
IF it's documented well, so explained.
Personally I prefer having two, or three lines of weird looking code, and fifty, sixty lines of comment, explaining it.
But that's the point: There are no fifty lines explaining it. Well, in my personal stuff there is. But that is not conform to any guideline I know. Most of the times there is nothing, explaining nothing, just the code only.
With someone who's versed in the language comments are not needed for large parts, would only trash up the place, lowering readability instead of increasing the understanding of the code. And better having no comments at all, then confusing ones. I fully agree. (Texteditors provide the possibility to blind out comments, if they are too distracting.)
Anybody did some more coding a bit beyond beginners classes knows:
Sometimes there are situations you have to do "weird things".
The reason may be often you simply won't come up with a better idea. But at least to me it seems, sometimes there actually is no other way than doing it weird. Those parts need to be documented - well!
A
well commented code is always faster to comprehend, than a code that's completely undocumented.
Because you don't need to reconstruct from the code what it's doing, and then
guess what the thoughts were the writer had why doing it this way. You get the ideas first, and then see how the code fits into that.
There are situations you look at some one elses code and think:"This idiot wrote some pretty crap here!" While I exclude real idiot's crap there are situations where the writer was no idiot at all, but had very good reasons to do it that way. What you may find out a lot later, when you are trying to rewrite this part better, but the shit simply ain't work otherwise than as it was written originally.
Because the author was no idiot at all.
She or he simply did not shared his ideas, thoughts, reasons, and conclusions.