This reminds me of an observation I’ve had: online when you see an expert tradesperson do something, they’ve got all kinds of specialized tools. Because they do the same kinds of things a lot and so those tools are worth buying and learning. But if you handed me those tools to do a job just once, I’d be much worse off than sticking to what’s in my workshop.
In the end we both have a finished project. But the professional would be a fool not to specialize their kit and I would be a fool if I did.
I wish I could follow this guys advice. My colleagues are Microsoft camp guys. C# on the back end for anything they can. If your problem can’t be solved with a windows desktop app on whatever .net framework is trending, then it’s not a problem worth considering.
But we needed a native app on iOS for irrigation control, so we had to tool up with Swift. I would love to do a solid cross platform thing, but there isn’t one in mobile space for user rich experiences that do Bluetooth, mapping, etc. We tried. So Kotlin has to be added to the mix so we could use the only right/available tool for native Android app job. The embedded stuff? Needed C code. The control node that runs our edge stuff needed to run on little Linux computers and support self hosted development. C wasn’t going to work, nor was Kotlin or Swift. Python it is. We needed a highly threaded cloud bridge sevice. Python was out for that. I guess I could have tried server Kotlin or Swift, but I went with Elixir instead and just wish I could use that everywhere.
So if you’re just doing web stuff, I guess you can just use consistent whatever. Though isn’t the whole argument of node/deno that since you have no (real) choice but to use JSON the front side, you might as well use it as the wrong tool for backend as well?
As a thought experiment, this got me thinking what would a solution just using .NET core look like? Obviously I don't have all the details but I think you can do all of those things in .NET (with the exception of the embedded stuff probably).
* Bluetooth and maps libraries are available for Xamarin, [1] [2]
* Control server could be written in C# using the .NET core worker template [3] and deployed as a SystemD service and then the SDK deployed as a package to enable local development.
* Cloud bridge deployed as a service or website using raw sockets, SignalR or Azure IOT hub (depending on requirements).
You'd end up with 2 languages instead of 5 and I suspect you'd be able to factor some code out into libraries as well...
> * Bluetooth and maps libraries are available for Xamarin, [1] [2]
I don’t have lots of direct experience with these in particular. I have played with the BLE shim libraries provided for Dart, React-Nativ, and Cordova back in the day. All were maybe ok for single characteristic rare interaction. But as you go up in connection frequency, characteristic count, or update frequency, things degrade in robustness quickly. I spent a solid couple of weeks getting our Kotlin/Android version up to being able to handle 1000 reconnects without hanging the chip. BLE is hard.
Given that even with the native map stuff, we had to jump through some “clever” hoops to get the kind of map overlay feedback we were looking for and at tolerable update speed, I remain skeptical that yet another abstraction layer wasn’t going to be an additional hindrance.
> * Control server could be written in C# using the .NET core worker template [3] and deployed as a SystemD service and then the SDK deployed as a package to enable local development.
We have limited flash space. It’s too small to accommodate the gcc toolchain. If/when we want to do that, we used a mounted sd card to accommodate the space needed for doing hosted development. It’s pretty cumbersome compared to having Python right there. I would guess that if gcc wasn’t much of an option, c# was going to run into similar issues.
> * Cloud bridge deployed as a service or website using raw sockets, SignalR or Azure IOT hub (depending on requirements).
My choice to use Elixir here would be the most arguable. The nature of how we do MQTT communications securely (uniquely secured by each connection path) dictates either a single threaded many-socket mqtt client (we’d have to do this from scratch) or just use lots of threads, one per client connection. It is not uncommon to have 20,000 threads active in our current solution. Based on some peers comments, that would be a lot. But at that point, I guess we’d pursue the many clients on less threads approach. Which could have been in whatever. I really just wanted an excuse to take Elixir for a spin, and this was an area it could/did shine.
Very interesting - thanks! It's an interesting exercise thinking through some of the constraints and some of the engineering tradeoffs. Out of interest, how much development is actually done on device vs being done on another system and then loaded onto the SD card?
> but I went with Elixir instead and just wish I could use that everywhere.
Same here! I would say, I'm in the similar camp as your colleagues but about Elixir. I'm glad to choose the right tool for the job as long as it's Elixir :)
The author repeatedly makes the case that people can handle more complexity when using familiar tools and that in many cases this can outweigh the costs of using familiar tools for tasks they’re poorly suited for. I completely agree!
It’s still a bit disappointing that there wasn’t any mention or consideration for becoming familiar with more tools as a solution.
You don’t need to be familiar with everything, but to extend the author’s metaphor it’s better to have a variety of tools in your toolbox than to be a one-trick pony.
There is you the individual and there is the group at work. I think this article is talking more about making decisions that impact the workplace.
Not all arguments between the individuals and the workplace translate because the workplace is a dynamic set of individuals. So any decision made for the workplace has to work for the people currently at work but also for future people you will onboard.
I embraced the fact that I know only one programming language at a decent level and it is Python. Rather trying to convince people that "I am a programmer", I convinced myself that, "I can solve problems through programming". I just love solving problems. No matter how hacky or stupid my solution is, it gets me from point A to point B. For many "programmers" doing things right gets in the way of doing things.
I have some bad news about programmers in every other mainstream language.
This confusion arises because “programming” is a broad, poorly-defined term. There are plenty of examples of projects which foundered because someone did what you described OR the inverse, because the underlying program is some combination of mismatched skills, incentives, and situations. There are people who’ve generated more business value with a labyrinthine Excel workbook than a room full of “real programmers”, and it’s always worth considering why.
mypy still has some rough edges but in a couple years it’s going to be golden. My Python programs are already always fully typed and checked with strict settings. For all intents and purposes that’s 95% of the way of what a statically typed language provides anyway (if we ignore multithreading).
I think OP is misunderstanding what "impedance" means? Impedance, at least in the context of electronics is describing resistance depending on frequency of the signal; if the incoming signal "vibrates" with a frequency that can make the receiver match it, energy transmission is large, otherwise low. Maybe I'm oversimplifying a bit, electronics or dynamic systems aren't my speciality. But the point is that two systems need to have the same impedance to move energy or a signal effectively. If the impedance matches, they "vibe" :)
So it's about matching the impedance. It's about reducing the impedance mismatch between systems, not about reducing the impedance.
I might actually go and track down and use the math on some example, and actually go through it myself for the first time--if I had the time. But I'm just barely non-busy enough to read HN at all right now.
Maybe wavelength of light and a filter is actually having the same thing happening underneath, and is used often in common talk: if two people "are of the same wavelength", they can talk to each other. The closer they are, the easier. That's not physical, of course, but is an analogy for physics: if the receiving system has a colour filter on its input, the light you're sending needs to be of that wavelength (or frequency, since that's directly related) or close enough to get through.
So if you want to maximise that two people are a match for each other (the "right tool"), you want to minimize the "difference in wavelength".
But then I think there is more to it than just the wavelength: it's also about the characteristics of the sending object. An amplifier and speaker pair needs to have about the same impedance for the amplifier to be able to output sound of a fair volume level through the speaker. Such systems are coupled bidirectionally (the speaker can divert the energy back into the amplifier), and that's where I think it gets complicated to a level where you and me zone out unless we study the math.
Rather than lazy programmers or evil managers, I think what primarily stops us keeping software simple is that we often seek to model complex systems, or code complex problems.
Given this, the “wrong tool (technology or language) for the job” simply complicates the already complex?
Every time I have let complexity live, or increased it in a program, it was due to deadlines, bugs in third party software, artificial complexity in third party interfaces, fear of touching a part that's a "ball of mud"... Rarely was it for technical reasons or because the problem domain was on the level of a PhD thesis - I would say they are "political" reasons. I think that's the daily life of the average programmer.
TFA talks about artificial vs essential complexity, with the term "impedance" being used to talk about artificial complexity introduced by the tool being used, typically the programming language. As the author notes, the tool being chosen is partly a political choice.
The paradox they talk about, is that e.g. doing CGI in C is "wrong", but on the other hand if you are extremely good at C, you might be more productive than using, say Perl. Between the lines of TFA, you read because productivity. The "wrong" tool is a local optimum (of your all-C shop). If you want to go after the global optimum, you need training. And that's another political decision - which can be tough to make, because as typically don't have expertise in the field, you don't know what's right and are likely to fall for fads or new hires that supposedly brings that expertise to you.
This reminds me of the 737 MAX. It was deliberately designed to have a flight profile very close to the original 737 so as not to require retraining pilots. That worked adequately, but not good enough.
(The MCAS system would have been different or even absent if pilot retraining was done anyway.)
Basically, they tried this approach but didn't recognise they crossed the threshold where letting go of old flight characteristics makes sense.
> Rarely was it for technical reasons or because the problem domain was on the level of a PhD thesis - I would say they are "political" reasons. I think that's the daily life of the average programmer.
Where I’ve seen gratuitous complexity introduced by the technical staff it’s been a situation where a 1.5x programmer thought they were being 10x when they larded up the architecture early, leaving the project behind schedule seemingly without enough time to fix it. The problem wasn’t the average programmer but letting the one a bit above average set the direction.
Technologists are better at understanding technology than business. If you are playing around with technology on your own dime, use the best tool for the job. If you are writing code professionally for a company, consider all the long term needs of the company. If you bring in a new language/framework/platform, they will have to staff for that. The default behavior should be to pick one of the most prolific technologies on the market and use that until it's no longer feasible.
I like this goes a level deeper than just the “keep it simple” mantra. Experienced developers know keeping complexity in check is critical but nevertheless keeping things simple is not always easy.
I'm in the process of rewriting an "accreted" codebase. The new one will be a lot more succinct, robust, customizable, localizable, accessible, and a hell of a lot faster.
But it is unlikely to be simpler. In fact, it is likely to have some fairly hairy "sleight of hand," to achieve its ends.
I've been at this a while. I know what I'm doing. Making the decision to rewrite the codebase, as opposed to fixing the existing one, was fraught. It was not a "snap" judgment at all. I agonized over it for some time. I'm sailing out of port[0]. There are circumstances that allow me to do this, and I'm grateful, but I am also setting myself up for a lot of work. A great deal of the "cruft," in the old codebase, is called "bug fixes." I'll have to deal with the same stuff that caused the patches. This time, I will be better able to anticipate them.
One of my favorite activities, when writing applications, is throwing away code. It gives me great pleasure to consolidate a bunch of classes or structs, into a common handler, and toss out all the unique files, etc.
Of course, one of my most powerful refactoring tools, when doing this, is OOP. In particular, polymorphism <cue a spooky rendition of Toccata and Fugue in D Minor>. It's an outstanding way to factor out common functionality. It is possible to use things like protocol/interface defaults to do this (and I use them frequently), but protocol defaults have their own issues[1].
It's highly unlikely that an inexperienced developer (especially one not fairly advanced in Swift) would be able to take over my codebase, despite the elegance of its design, and its effectiveness. I would likely hear cries of "overengineering!", and statements like "Why didn't they do this in React Native, or Xamarin?", or "I could bang the same thing out in SwiftUI, in two days!", etc. ad nauseam.
Fair 'nuff, but I an an advanced Native Swift developer. It's the tool I know best, and I know it better than most. I have also been severely burned, in many ways, by being too "bleeding edge," and playing Buzzword Bingo. I know of several major projects, where the company decided to write their next version, using the latest tools, only to have to clean up the mess, and revert to the classics, when it all went pear-shaped.
I'm glad Hillel Wayne is getting into this, but as usual I am highly skeptical of the solution he is jesticulating towards.
The problem is not too many programming languages lol. How the hell could the difference between 15 and 3 be the problem!
The problem is the terrible division of laber between projects, between libraries, between kernel space and user space, and between hardware and software.
The problem is Conways law, or zoomed out, incremental evolution on the industry scale promotes biology-like over-complexity.
The solution is planning, and the first step of planning is seeing what the hell is going on.
The key ingredient for that first step is Nixpkgs.
Yet another software opinion article preaching mediocrity. Imagine a carpenter that only uses a hammer because learning other tools is too hard for them, and he'd have to bring his colleagues up to speed, and ah forget it we can use nails instead of screws anyway.
"Oh! Drain pipe for the faucet? I know, I'll just use leftover air vent ducts, that's what I'm already familiar with anyway!" (Using nails to join everything of course, how's that for a leaky abstraction?)
As he proclaims the "first sprint" done he walks up to you, demanding a 100K/year salary. When you decline, because the house is falling apart, he angrily mutters something about how "tech debt" is wrecking this industry (he's blaming the failure on you for not giving enough time to address it) and that he was thinking about finding another job anyway, since he was already working for you close to a year now.
The point isn't to use a hammer for everything, it's that you maybe don't need to get a new, slightly different sized, hammer every time you get a slightly different size of screw to hit in. And that when you want to pull a nail out you can use the other side on your claw hammer, rather than go hunting for a dedicated NailRemoverPro TM that you've heard will make your nail removal even smoother.
Unlike with carpentry, when it comes to programming languages there actually are multiple options that can be used for such a wide range of purposes that it's possible to built entire projects/products using just the single one you prefer, which is of course less likely to be feasible when it comes to using a single tool like a hammer.
Choice of language has much larger implications than you are implying. Yes, every language is turing complete. Now write a regex engine in pure "turing machine". Sucks, doesn't it? This is known as the "Turing Tarpit".
Every language has unique strengths and weaknesses and it's absurd to pretend they're all kinda the same. I wouldn't write a website in C. I wouldn't write a robot arm driver in PHP. I wouldn't write anything more complicated than a throwaway script in bash.
I don't mean to be disrespectful, and perhaps I didn't read your reply right. But it seems to me if you don't differentiate languages, then you either don't have enough knowledge of and exposure to various programming techniques and language design ideas, or you're making an argument in bad faith.
I have written a web site in C about twenty years ago, and it was terrible. Absolutely Turing Tarpit category, the language is just completely inadequate for this problem and so you spend all your effort trying to build something which is suitable out of the language, which again, yes, Turing Completeness, of course it's technically possible but it's a bad idea.
Web sites are full of strings, and C doesn't really grok strings. Not in the way WUFFS doesn't grok strings (there are no strings in WUFFS, it has no string type, it's not a general purpose language it's not for that), more like the way New Forest ponies don't grok traffic laws.
Many high-level languages with good string support are implemented in C. The C standard library sucks but it's not your only option for handling strings.
This isn't a fair analogy. There's a cost to adding new tools to a software environment, and an ongoing cost to maintaining that tool. Closer, IMO, is you're speccing out a new house with a builder and ask for a swimming pool, a hot tub, jacuzzi tubs, and body sprayers in the showers. Getting all of those installed and running to spec is going to be much more expensive than getting one or two of them. If your builder says "You really need the pool. I'll see if I can get that dug in my free time, don't worry about it." you are probably going to worry about what you're actually getting. If you do have the budget to do everything properly, you also need to have the ongoing budget for cleaning and maintenance. Pools and tubs have very different maintenance profiles. The same is true for software.
Which is to say, using the "right" tool is nice, but not always in the budget. R or Julia may be a better language for data science than Python, but if you're the only one using them, you're just making things harder for the rest of your team. A Python shop has probably invested in coding standards, linters, test infrastructure, common libraries, and has expectations for how code runs "in production". Expect that you'll need to clear at least some of that bar for a new tool you introduce. Expanding towards tools that better fit problems is good, but unless you're doing 100% green field development it needs to be intentional and measured.
That's not what I took away from the article. On the contrary, to me, the article listed several trade-offs that come with choosing what might be the "right" tool at any given time.
The carpenter metaphor does not really fit here, because at some point, a carpenter is clearly "done" and moves on to other projects, whereas developers continuously maintain and extend the work - their own and their colleague's.
When what you have built is effectively a bike, and the customer requests 2 additional wheels, you don't duct tape them on. You rewrite from scratch, building the customer a car this time.
Sometimes, you can salvage parts. Maybe the frame is still good when the customer reveals they actually want a minivan instead of a car. Maybe you can reuse the engine and wheels and transmission and seats. But the other stuff has got to be replaced.
Why is our industry so against rewriting? There seems to be a massive focus on building stuff so generic such that it is magically extendable into anything. Hint: it never is. It never will be.
Write specialized code, which is always smaller, clearer, easier to understand code against the heavily abstracted bullshit you'd otherwise get.
And yes, sometimes that means the "bicycle wheel" library/framework/language needs to be replaced in your shop.
This ignores the cost of replacing code. I agree with your point that you should focus on the problem at hand, and not make code too generic, but if you only make specialized code with the idea of “we can rewrite or replace it later” you will end up with a lot of specialized code that is never rewritten.
You stare in horror at your new GoLand house with GoLand(tm) roof, floor, tiles and windows. At least it isn't Node you reassure yourself as the AWS bills roll in.
I don’t understand this comment one bit. GoLand is a decent IDE. Go is a fairly fast compiled language, likely to produce faster code than the average AWS workload.
In the end we both have a finished project. But the professional would be a fool not to specialize their kit and I would be a fool if I did.