i uh spend a lot of time thinking about whether various surprising software design choices are
a) intrinsic to the problem domain ("it turns out it DOES make sense!”)
b) made sense historically ("this made sense in 1992, but it didn't age well”)
c) just a typo/mistake (the "Referer" header)
d) related to budget/time constraints (“well, prototyping with shell scripts is fast!”)
e) cultural/organizational (“well, Google is the main funder for this project, and…”)
f) something else
a lot of people are commenting with additional reasons for surprising design choices like “malice", "incompetence", “stupidity", ”didn't care enough to do it well"
I suppose those come up sometimes but I intentionally didn't mention them because when I think about major technologies we use (SQL, git, DNS, bash, etc), the reasons I find that they're a little weird are usually pretty legitimate and not like "well the people who built them were stupid”
@b0rk you are absolutely right, but "didn't think of this" encompasses things like "didn't foresee two billion mobile phones each individually more powerful than all existing-at-the-time computers combined". DNS is not great for dealing with that part of the network.
Ignorance is not a character flaw, it's just a fact.
@b0rk The late Gerald Weinberg had a "law" for this sort of thing, which he called Boulding's Backwards Basis: "Things are the way they are because they got that way."
Surprising design choices often arise from a succession of smaller decisions, each of which made perfect sense in context at the time. In hindsight it's easy to say that decision X was a mistake, when at the time X was made, it clearly wasn't, or the risk of a bad outcome was considered acceptable.
@b0rk there’s also an interesting tendency to make it personal: we say “the person who wrote this was an idiot”, not “worked for a company which put inexperienced people in stressful situations with inadequate support”.
Back in the 90s, I had to salvage a failing project. At first I blamed the guys writing loops instead of SQL JOINs or copy-pasting because they didn’t know how to write functions, but then realized the company billing them out as senior at $250/hour was the real villain.
@b0rk Consider the reasons that IPv4 (and IPv6 too) have no functional security layer (authenticated headers, encrypted payloads, etc.) on the public Internet. IPsec works only on virtual private networks, despite the clear public interest in the welfare to be provided by a public Internet with a working security layer. There are discoverable reasons for it, and none of them are in your “mostly legitimate” categories.
@janxdevil@b0rk Is there more to this story than "When IPv4 was devised it was connecting machines behind locked doors, and when IPv6 came along they tried to do an incremental improvement on IPv4 and adding a whole security protocol would have been a bridge too far?"
@mark@janxdevil@b0rk It seems that for most IPv6 is a bridge too far as-is, let alone with a security protocol on top of it.
It's also the case that a lot of the folks who were influential were also privacy absolutists, so building in rigid traceability would have been an anti-priority.
Plenty of people want Tor to be the Internet, and a lot of those folks write software.
@b0rk So many things gs seem obvious in hindsight, and with fresh eyes to look at the problem. But when it was a NEW problem that somebody was trying to solve, and with the tools and patterns available AT THE TIME they were solving it, it's often a very different story.
Which is why I try to give these things the benefit of the doubt.
@mwop@b0rk I also don’t usually think it’s “stupid people” (I did in the past; it was a naïve opinion mostly). I do think that we try to build software much faster and much cheaper than it should be done, and that leads to a lot of the faults mentioned above.
Historically when we’ve accepted this kind of corner-cutting in other fields, it’s been dangerous, and now we spend on efforts like “bridge engineering”.
…but software has become that kind of dangerously prevalent.
a few additions to this "list of reasons for surprising software design choices" from the replies
g) "this was built for a very specific use case and then a lot of people started using it for completely different things” (would love examples of this one)
h) an arbitrary decision early on had unexpected consequences (would love examples of this one)
i) we needed to learn the hard way (for example, the way TLS was designed through finding vulnerabilities and redesigning to mitigate)
@b0rk I maintained an in-house 3D engine once. It had a full scene graph, but most of what we did with it were 2D image buttons - but we worried a lot about 3D.
Any library that has only one user can be an example in some sense - and the artifacts are build times, process complexity and linkage costs.
@b0rk hey, I thought of one that might be (b) but might be different: it was designed to work under resource constraints that made sense historically but that no longer apply (and may be forgotten).
Examples:
Horrible sendmail.cf syntax was designed to be parsred very quickly on slow 1970's hardware with not-fully-developed-yet 1970's parsing technology.
C90 standard requires that externally-visible identifiers (e.g. library functions like printf and malloc) must have nanes that are all unique in the first 6 characters, for compatibility with then-existing linkers that ran in highly memory-constrained environments.
@mjd@b0rk I often think that the C preprocessor as a whole belongs in that category.
It approximates features of more modern languages and compilers – inlining and generic programming, in particular - which would have been too expensive to do in the 1980s the way they're normally done these days (generics via sophisticated type systems, inlining in the optimiser).
cpp doesn't do them very well by today's standards, but it does them cheaply enough to be feasible in the 80s.
@b0rk g) would be PHP probably. Started as scriptable template engine and ended as a de facto language for web for a quite time.
h) PHP again - some naming decisions stay to this day, for example because early PHP implementations used some utterly dumb hashing function for function names (I do not know if that was length + sum of characters or something like that, but it was dumb).
An IMO common variant of (b) is "this thing existed for a totally different reason and was good enough here". I mention this explicitly, because it's a variant of (b) that can happen without ~any passage of time.
Same. I spend a huge amount of time figuring out WHY things are the way they are.
Another reason I've found surprisingly often is "It wasn't supposed to be this successful, this was a personal toy and it took off." OpenSSL is the textbook example here.
Add comment