I am personally of the opinion that ML will end up being 'normal technology', albeit incredibly transformative.
I think you can combine 'Incanters' and 'Process Engineers' into one - 'Users'. Jobs that encompass a role that requires accountability will be directing, providing context, and verifying the output of agents, almost like how millions of workers know basic computer skills and Microsoft Office.
In my opinion, how at-risk a job is in the LLM era comes down to:
1: How easy is it to construct RL loops to hillclimb on performance?
2: How easy is it to construct a LLM harness to perform the tasks?
3: How much of the job is a structured set of tasks vs. taking accountability? What's the consequence of a mistake? How much of it comes down to human relationships?
Hence why I've been quite bullish on software engineering (but not coding). You can easy set up 1) and 2) on contrived or sandboxed coding tasks but then 3) expands and dominates the rest of the role.
On Model Trainers -- I'm not so convinced that RLHF puts the professional experts out of work, for a few reasons. Firstly, nearly all human data companies produce data that is somewhat contrived, by definition of having people grade outputs on a contracting platform; plus there's a seemingly unlimited bound on how much data we can harvest in the world. Secondly, as I mentioned before, the bottleneck is both accountability and the ability for the model to find fresh context without error.
> I think you can combine 'Incanters' and 'Process Engineers' into one - 'Users'
I wanted to talk about this more but couldn't quite figure out how to phrase it, so I cut a fair bit: with "incanters" I'm trying to point at a sort of ... intuitive, more informal practitioner knowledge / metis, and contrast it with a more statistically rigorous approach in "statistical/process engineers". I expect a lot of people will fuse the two, but I'm trying to stake out some tentpoles here. Users integrate a continuum of approaches, including individual intuition, folklore, formal and informal texts, scientific papers, and rigorously designed harnesses & in-house experiments. Like farming--there's deep, intuitive knowledge of local climate and landraces, but also big industrial practice, and also research plots, and those different approaches inform (and override) each other in complex ways.
In some sense, technology is "not normal" regardless.
If we think of the digitization tech revolution... the changes it made to the economy are hard to describe well, even now.
In the early days, it was going to turn banks from billion dollar businesses to million dollar ones. Universities would be able to eliminate most of their admin. Accounting and finances would be trivialized. Etc.
Earlier tech revolution s were unpredictable too... But at lest retrospectively they made sense.
It's not that clear what the core activities of our economy even are. It's clear at micro level, but as you zoom out it gets blurry.
Why is accountability needed? It's clearly needed in its context... but it's hard to understand how it aggregates.
Accountability is really a way to address liability. So long as people can sue and companies can pay out, or individuals can go to jail, there is always going to be a question of liability; and historically the courts have not looked kindly at those who throw their hands up in the air and say “I was just following orders from a human/entity”
This is dependent on having a court system uncaptured by corruption. We're already seeing that large corporations in the "too big to fail" categories fall outside of government control. And in countries with bribing/lobbying legalized or ignored they have the funds to capture the courts.
A huge component of compulsory (either by statute or de-facto as a result of adjacent statute, like mandatory insurance + requirements thereof) professional licensure is that if you follow the rules set by (some entity deputized by) government the government will in return never leave you holding the bag. The government gains partial control and the people under it's control get partial protection.
"oh I'm sorry your hospital burned down mr plantiff but the electrician was following his professional rules so his liability is capped at <small number> you'll just have to eat this one"
I would wager that a solid half if not more of the economy exists under some sort of arrangement like that.
> Hence why I've been quite bullish on software engineering (but not coding). You can easy set up 1) and 2) on contrived or sandboxed coding tasks but then 3) expands and dominates the rest of the role.
Why can't LLMs and agents progress further to do this software engineering job better than an actual software engineer? I've never seen anyone give a satisfactory answer to this. Especially the part about making mistakes. A lot of the defense of LLM shortcomings (i.e., generating crappy code) comes down to "well humans write bad code too." OK? Well, humans make mistakes too. Theoretically, an LLM software engineer will make far fewer than a human. So why should I prefer keeping you in the loop?
It's why I just can't understand the mindset of software engineers who are giddy about the direction things are going. There really is nothing special about your expertise that an LLM can't achieve, theoretically.
We're always so enamored by new and exciting technology that we fail to realize the people in charge are more than happy to completely bury us with it.
It's not about whether they make mistakes (they do! although the exact definition of a mistake is nuanced), but whether they can take accountability if the software fails and millions are lost or people die. A large part of the premium paid on software engineers is to take accountability for their work. If a "business person" directs their agent to build some software and takes accountability -- congrats! They are also now a software engineer :)
The lines between a software engineer / business person / product / design and everything else will blur, because AI increases the individual person's leverage. I posit that there will be more 'software engineers' in this new world, but also more product people, more business people, more companies in general.
> Why can't LLMs and agents progress further to do this software engineering job better than an actual software engineer?
Because a machine can never take accountability. If a software engineer throughout the entire year has been directing AI with prompts that created weaker systems then that person is on the chopping block, not the AI. Compared to another software engineer who directed prompts to expand the system and generate extra revenue streams.
> Because a machine can never take accountability.
A business leader can though.
> Compared to another software engineer who directed prompts to expand the system and generate extra revenue streams.
I think you're missing the point. Why can't an LLM advance sufficiently to be a REAL senior software engineer that a business person/product manager is prompting instead of YOU, a software engineer? Why are YOU specifically needed if an LLM can do a better job of it than you? I can't believe people are so naive to not see what the endgame is: getting rid of those primadonna software engineers that the C-suite and managers have nothing but contempt for.
If a 'business leader' is prompting out software through their agents, ensuring it works, maintaining it, and taking accountability... they're also a software engineer
That's how things work already in every workplace where there's any real danger. The companies construes its policies and paper trail in bad faith so that the employees are always operating contradictory to policy/training and then when something happens blame can be shifted on them.
> It's why I just can't understand the mindset of software engineers who are giddy about this brave new world. There really is nothing special about your expertise that an LLM can't achieve, theoretically.
They’re stupid or they’re already set up for success. The general ideas seems to be generalists are screwed, domain experts will be fine.
An enormous amount of domain expertise is not legible to LLMs. Their dependence on obtaining knowledge through someone else's writing is a real limitation. A lot of human domain expertise is not acquired that way.
They still have a long way to go before they can master a domain from first principles, which constrains the mastery possible.
I think these arguments tend to reach impasse because one gravitates to one of two views:
1) My experiences with LLMs are so impressive that I consider their output to generally be better than what the typical developer would produce. People who can't see this have not gotten enough experience with the models I find so impressive, or are in denial about the devaluation of their skills.
2) My experiences with LLMs have been mundane. People who see them as transformative lack the expertise required to distinguish between mediocre and excellent code, leading them to deny there is a difference.
Not sure that's what I was getting at. People in camp 2 don't think an LLM can take over the job of a real software engineer.
It's people in camp 1 that I wonder about. They're convinced that LLMs can accomplish anything and understand a codebase better than anyone (and that may be the case!). However, they're simultaneously convinced that they'll still be needed to do the prompting because ???reasons???.
I was at 2) until the end of last year, then LLM/agent/harnesses had a capability jump that didn't quite bring me to be a 1) but was a big enough jump in that direction that I don't see why I shouldn't believe we get there soonish.
So now I tend to think a lot of people are in heavy denial in thinking that LLMs are going to stop getting better before they personally end up under the steamroller, but I'm not sure what this faith is based on.
I also think people tend to treat the "will LLMs replace <job>" question in too much of a binary manner. LLMs don't have to replace every last person that does a specific job to be wildly disruptive, if they replace 90% of the people that do a particular job by making the last 10% much more productive that's still a cataclysmic amount of job displacement in economic terms.
Even if they replace just 10-30% that's still a huge amount of displacement, for reference the unemployment rate during the Great Depression was 25%.
I think the reason AI isn't going to replace CEOs, or anyone in the C suite, is pretty obvious. They see themselves as the company. Everyone else is a resource. AI is here to replace resources, just like investing in a brand new lawn mower. For them, replacing an executive with AI is like saying you're going to marry a broom.
That's true, too. I guess we will see if executive pay and credentials start going down. They could technically have AI make all the decisions while someone just plays the patsy.
They're just a thin layer to be replaced last. They're just arrogant enough to think they're the company, but ultimately the endgame is -- all humans become economically insignificant compared to the automated economy.
The problem with AI is that it isn't like any previous technology. There may be temporary jobs to fill in the gaps but they won't be careers. The AI will do the process engineering and self optimization. The prompt witchcraft is a good example because today its totally unnecessary and doesn't actually increase performance, and they'll continue to make it easier to direct/steer the models.
We're literally trying to build an intelligence to replace us.
The human species. "We" doesn't include everyone and doesn't necessarily imply the process happens through collaboration and planning (conspiracy). The race to automation is happening as expected; outside any group control and bound by competition. Game theory suggests the end result is us being replaced, if we make it that far. "We" as a species are the ones making it happen.
Loved that section about "meat shields". LLMs cannot be held accountable. Someone needs to be involved in decision making, with real stakes if those decisions are bad.
It just makes logical sense really; the human using the tool is in the end responsible.
Whether the tool is too powerful or ethical to use is an orthogonal discussion, in my opinion. Taken to the extreme, nuclear weapons still need someone fire or drop them. (We should still have discussions on safety and ethics always!)
why can't the name be 'scape goat'? Since that's what they are - the "real" responsibility rests on the owners, and they happily shed it as limited liability ownership of shares.
All plausible, but not very transformative. Like imagining that the new jobs enabled for the automobile include automobile maintenance, tire shops, and so on. Traveling nurses, motel operators, military tanks, doordash, suburban life, beer sales at NASCAR, those were all enabled by the car (and its larger sibling the truck).
Still missing are the jobs snd industries enabled by "AI" that are not themselves "AI".
I think that this is an interesting attempt at taxonomy, but it's a bit on the magical thinking end (and I say this as somebody that does a good amount of what's described as the incanter role). It's a combination of the author's previous witchy aesthetic (see his excellent "<x>ing the technical interview" series) and progressive labor politics (which are asymptotically doomed in the current automation push).
The biggest failure of imagination, I think, is the assumption we'd use humans for most (or *any) of these jobs--for example, the work of the haruspex is better left to an LLM that can process the myriad of internal states (this is the mechanical interpretation field).
Yes, I had the same impression. I'm sympathetic to the author's perspective but I can't muster even the minimal optimism they've shown here. The "process engineers" as described would themselves quickly be replaced by an automated system. The "statistical engineers", I think, would never be able to keep up with the rate of change of the AI models, which would likely have different statistical behavior and biases in each language/context/etc with each update, and so it's unlikely anyone would pay them to develop that required deep expertise in the first place. More likely, that work would be done at an AI foundation model company -- but it would be done just once, and then incorporated into the training process.
As an engineer, I'm never more excited about this job.
My implementation speed and bug fixing my typed code to be the bottleneck - now I just think about an implementation and it then exist - As long as I thought about the structure/input/output/testability and logic flow correctly and made sure I included all that information, it just works, nicely, with tests.
Unix philosophy works well with LLM too - you can have software that does one thing well and only one thing well, that fit in their context and do not lead to haphazard behavior.
Now my day essentially revolves around delivering/improving on delivering concentrated engineering thinking, which in my opinion is the pure part about engineer profession itself. I like it quite a lot.
Though something I half-miss is using my own software as I build it to get a visceral feel for the abstractions so far. I've found that testability is a good enough proxy for "nice to use" since I think "nice to use" tends to mean that a subsystem is decoupled enough to cover unexpected usage patterns, and that's an incidental side-effect of testability.
One concern I have is that it's getting harder to demonstrate ability.
e.g. Github profiles were a good signal though one that nobody cared about unless the hiring person was an engineer who could evaluate it. But now that signal is even more rubbish. Even readmes and blog posts are becoming worse signals since they don't necessarily showcase your own communication skills anymore nor how you think about problems.
Funny enough, I think github and communication are still a huge part of what I see.
Github code itself maybe irrelevant, but is the product KISS/UNIX? Or is it an demonstration of complete lack of discipline about what "feature" should be added. If you see something that have multiple weakly or completely irrelevant feature strung together, it's saying something. Additionally, AI would often create speghetti structures, and require human shepherding to ensure the structure remain sound.
Same with communication. I have AI smell, I know if something is AI slop. In my current job, docs sent with expectation for others to read always prefaced with -- this section typed 100% by aperocky -- and I dispensed with grammar and spelling checks for added authenticity. I'll then add -- following section is AI generated -- to mark the end of my personal writing.
I think that is the way to go in the future. I pass intentional thinking into AI, not the other way around. There are knowledge flowing back for sure, but only humans possess intention, at least for now.
> But now that signal is even more rubbish. Even readmes and blog posts are becoming worse signals since they don't necessarily showcase your own communication skills anymore nor how you think about problems.
Yup. I've spotted former coworkers who I know for a fact can barely write in their native language, let alone in English, working for AWS and writing English-language technical blog posts in full AI-ese. Full of the usual "it's not X, it's Y", full of AI-slop. Most of the text is filler, with a few tidbits of real content here and there.
I don't know before, but now blog posts have become more noise than signal.
> My implementation speed and bug fixing my typed code to be the bottleneck
I remember those days fondly and often wish I could return to them. These days it's not uncommon to go a couple days without writing a meaningful amount of code. The cost of becoming too senior I suppose.
Anecdotally I've been observing a significant uptick in the amount of code being produced by my peers who are in senior engineer, leadership and engineering management positions.
They can take their 20+ years of experience and use it to build working systems in the gaps between meetings now. Previously they would have to carve out at least half a day of uninterrupted time to get something meaningful done.
> As an engineer, I'm never more excited about this job.
How long do you think it'll take for the AI trend to mostly automate the parts of your job that still make you excited?
Everyone thinks it won't be them, it will be others that will be impacted. We all think what we do is somehow unique and cannot be automated away by AI, and that our jobs are safe for the time being.
As someone in 99th percentile in terms of token usage, it's super clear to me where the agent will not be able to replace my judgement, two areas:
1. if it exceed the context the agent does random stuff, that are often against simplicity and coherent logical structure.
2. LLM has zero intention, and rely on you to decide what to build and more importantly not build.
As such, I'm the limit of the numbers of concurrent agents working fo rme, because there is still a limit to my output of engineering judgement. I do get better, both at generating and delivering this judgement. Exceeding this limit, the output becomes garbage.
At this current year and date, the AI does not automate me in anyway, I have something that they just flat out don't have.
Playing devil's advocate here, I'm not antagonizing you but thinking out loud.
> if it exceed the context the agent does random stuff, that are often against simplicity and coherent logical structure.
That's a current technical limitation. Are you so sure it won't be overcome in the near/mid future?
> LLM has zero intention, and rely on you to decide what to build and more importantly not build
But work is being done to even remove or automate this layer, right? It can be hyperbole (in fact, it is) but aren't Anthropic et al predicting this? Why wouldn't your boss, or your boss' boss, do this instead of you? If they lack the judgment currently, are you so sure they cannot gain it, if they don't have to waste time learning how to code? If not now, what about soon-ish?
> At this current year and date, the AI does not automate me in anyway
Not now, granted. But what about soon? In other words, shouldn't you be worried as well as excited?
> How long do you think it'll take for the AI trend to mostly automate the parts of your job that still make you excited?
Yeah, no one ever thinks beyond "whoa, how cool, I cloned Slack in 15 minutes!"
Personally, the thing I find more depressing is turning a career that was primarily about solving interesting puzzles in elegant ways into managing a swarm of idiot savant chatbots with "OK, that looks good" or "no, do it better" commands.
The problem that I'm trying to solve with agent is similar here, for instance, my comment likely made zero impression on you because I'm against both of the things that you are also against here.
I don't understand the title. It doesn't seem exactly clickbait but also doesn't seem to be what the article is about?
Anyway: The new job types might seem overspecialized now but history shows us this is indeed what happens as new industries open up. I think these predictions look quite solid.
Geo blocking the UK satisfies any age verification, otherwise the site owner would have to check if their content is considered adult in the UK and implement something.
> IMO a small blog website is not going to get pulled-up for this
Well, maybe not the typical engineering blog but I think if you're a puritan some posts/texts from Aphyr probably reaches borderline "adult content", so I'm not that surprised Aphyr rather play it safe and also make a point at the same time.
the culture section of this writeup links to explicitly adult/erotic content in the footnotes and discusses 'adult themes' directly. his caution seems reasonable.
It's probably a political point, but I think your comparison over sells how inconvenient it is for someone to geoblock one small country and the headache if anything did happen. It's not much more effort than doing nothing really?
And clearly users in the UK can find their own way to read it if they like, so the cost is also small there.
Considering that there is multiple "why is this blocked in the uk" comments on every single one of these posts maybe the UK isn't such a small country. Geoblocking a decent chunk of your readership would be a pretty big inconvenience for a writer I would imagine.
Have you even read the shit politicians are either pulling or trying to these days? There is no amount of paranoia that is too little when talking about things like cross national prosecution, laws regarding users not considered adults, and age verification.
Never know when one of your posts might gain serious traction. Not worth the risk. Very easy to find many examples of people making decisions thinking “I/we will never be big enough for that to be relevant” only to be haunted by that decision later. Classic example: partnership agreements/contracts between friends and family on small endeavors.
It's self-imposed, I think? curl connects to the same aphyr.com in both cases, but when connecting from the UK it receives a different response body. Probably sensible I expect, if you just want things to work, legally speaking.
Can people make a soft assumption that if somebody went through the trouble of digging up an archive link, then access to the article is limited in some way?
Humans will be held accountable, not machines, whatever is the technology used. The jobs you suggest are based on the state of LLM right now, this could change rapidly, considering the state of progress. These are just activities that are already done by people that work with these instruments, because they want to optimize and obtain the best/safest output from these machines.
> Humans will be held accountable, not machines, whatever is the technology used
Isn't this addressed explicitly in TFA, in section "meat shields"?
As for the rest, if you predict even the jobs described in TFA will be obsoleted by future LLMs+tools, then the future is even more dire than predicted by Aphyr, right? Fewer jobs for humans to do.
This is part 9 of a 10-part series. The author has posted every chapter to Hacker News every day for the past 9 days. Every time four of the first five or so comments are:
Someone noting it is unavailable in the UK.
Someone posting an archive.is link.
Someone asking why the above posted an archive link to a static site.
An answer that it is because the content is otherwise unavailable in the UK.
Do we really need to see this every single time?
I realize I am also not adding to the real discussion now as well, but Jesus Christ, this is irritating. Can we get a new rule that an author posting their own content, knowing it is unavailable in the UK, has to post their own archive link and explain why they're doing so as part of the submission?
>Can we get a new rule that an author posting their own content, knowing it is unavailable in the UK, has to post their own archive link and explain why they're doing so as part of the submission?
[Author blocks link to avoid being potentially in violation of the law]
You ask author to willingly provide link to again potentially be in violation of the law
So you are thinking that the UK government is going to do an international criminal investigation against aphyr for posting an archive link on a hacker news thread.
Does the UK government have the legal right to do an international criminal investigation against any website that is potentially violating their laws by having visitors from the UK accessing the site?
Answer yes or no, this is an easy binary question, and not one that requires any probabilistic thinking.
A company like Amazon doesn't treat its warehouse workers as human beings. Workers are seen as disposable: forced to piss in bottles, forced to work around the corpses of their collapsed coworkers, paid the absolute minimum possible, and replaced the second they don't operate like a perfect unfailing machine. You aren't viewed like a human, you are a tool. Cattle. A piece of meat they are forced to retain because a robot isn't quite capable of doing your task yet.
The article's use of "meat shields" isn't any different. Humans are going to be hired for the sole reason of taking accountability for actions dictated by AI. They are there only because the company can't put blame on a machine and will be sued to oblivion if there's nobody to blame at all. Your existence as a person is irrelevant, they are just interested in someone with a heartbeat they can blame when stuff inevitably goes wrong.
> someone with a heartbeat they can blame when stuff inevitably goes wrong.
if said person can be blamed (and take on the liability), but cannot stop the action or audit the action, take preventative measures (which costs money) etc, then they cannot take responsibility for real and thus whether the blame falls on them on paper is irrelevant - if there's real punishment (like jail time), but no real power to enforce anything, then who would be stupid enough to take on this job? If there's no real punishment, then what does it matter that the blame on paper is there?
Someone who needs to feed their family given that AI CEOs predicting that their technology will either destroy the world, take everyone's jobs or both.
In the article, Ctrl+F for "meat" returns 3 results, while "human" returns 8. Seems like "human" remains the dominant word of choice in this author's vernacular.
Edit: Further, the only times "meat" appears is in the phrase "meat shield", which is an analogy that is very apt relative to the crux of the article.
There has always been a subset of highly technical people in the software world who are anti-organic. They dislike the "meatspace" and humans and relate more with machines and software.
Yeah, that was what I was refering to, no the specific part of the article. I've seen it much more here recently. Kind of disgusting and sad, but on the other hand it's good if people show their real face that way.
I think you can combine 'Incanters' and 'Process Engineers' into one - 'Users'. Jobs that encompass a role that requires accountability will be directing, providing context, and verifying the output of agents, almost like how millions of workers know basic computer skills and Microsoft Office.
In my opinion, how at-risk a job is in the LLM era comes down to:
1: How easy is it to construct RL loops to hillclimb on performance?
2: How easy is it to construct a LLM harness to perform the tasks?
3: How much of the job is a structured set of tasks vs. taking accountability? What's the consequence of a mistake? How much of it comes down to human relationships?
Hence why I've been quite bullish on software engineering (but not coding). You can easy set up 1) and 2) on contrived or sandboxed coding tasks but then 3) expands and dominates the rest of the role.
On Model Trainers -- I'm not so convinced that RLHF puts the professional experts out of work, for a few reasons. Firstly, nearly all human data companies produce data that is somewhat contrived, by definition of having people grade outputs on a contracting platform; plus there's a seemingly unlimited bound on how much data we can harvest in the world. Secondly, as I mentioned before, the bottleneck is both accountability and the ability for the model to find fresh context without error.
reply