- cross-posted to:
- programming@programming.dev
You must log in or # to comment.
@dgerard What fascinates me is *why* coders who use LLMs think they’re more productive. Is the complexity of their prompt interaction misleading them as to how effective the outputs it results in are? Or something else?
What fascinates me is why coders who use LLMs think they’re more productive.
As @dgerard@awful.systems wrote, LLM usage has been compared to gambling addiction: https://pivot-to-ai.com/2025/06/05/generative-ai-runs-on-gambling-addiction-just-one-more-prompt-bro/
I wonder to what extent his might explain this phenomenon. Many gambling addicts aren’t fully aware of their losses, either, I guess.
The reward mechanism in the brain is triggered when you bet. I think it also triggers a second time when you do win, but I’m not sure. So, yeah, sometimes the LLM spits out something good, and your brain rewards you already when you ask it. Hence, you probably do feel better, because you constantly get hits dopamine.
Most people want to do the least possible work with the least possible effort and AI is the vehicle for that. They say whatever words make AI sound good. There’s no reason to take their words at face value.
Here’s a random guess. They are thinking less, so time seems to go by quicker. Think about how long 2 hours of calculus homework seems vs 2 hours sitting on the beach.
This is such a wild example to me because sitting at beach is extremely boring and takes forever whereas doing calculus is at least engaging so time flies reasonably quick.
Like when I think what takes the longest in my life I don’t think “those times when I’m actively solving problems”, I think “those times I sit in a waiting room at the doctors with nothing to do” or “commuting, ditto”.
I know what you mean. If I’m absorbed in something I find interesting time flies. Solving integrals is not one those for me.
Software and computers are a joke at this point.
Computers no longer solve real problems and are now just used to solve the problems that overly complex software running on monstrous cheap hardware create.
“Hey I’d like to run a simple electronics schematic program like we had in the DOS days, it ran in 640K and responded instantly!”
“OK sure first you’ll need the latest Windows 11 with 64G of RAM and 2TB of storage, running on at least 24 cores, then you need to install a container for the Docker for the VM for the flatpak for the library for the framework because the programmer liked the blue icon, then make sure you are always connected to the internet for updates or it won’t run, and somehow the program will still just look like a 16 bit VB app from 1995.”
“Well that sounds complicated, where’s the support webpage for installing the program in Windows 7?”
“Do you have the latest AI agents installed in your web browser?”
“It’s asking me to click OK but I didn’t install the 1GB mouse driver that sends my porn browsing habits to Amazon…”
“Just click OK on all the EULAs so you lose the right to the work you’ll create with this software, then install a few more dependencies, languages, entire VMs written in byte code compiled to HTML to run on JAVA, then make sure you have a PON from your ISP otherwise how can you expect to have a few kilobytes of data be processed on your computer? This is all in the cloud, baby!”
And generate shit code
I just want to point out that every single heavily downvoted, idiotic pro-AI reply on this post is from a .ml user (with one programming.dev thrown in).
I wonder which way the causation flows.
Machine learning is essentially AI with a paper-thin disguise, so that makes sense
It’s kind of the opposite, GenAI is downstream of machine learning which is how artificial neural networks rebranded after the previous AI winter ended.
Also after taking a look there I don’t think lemmy.ml has anything in particular to do with machine learning, it looks more like a straight attempt at a /r/all clone.
the ml in lemmy.ml stands for marxism-leninism
wait til you find out what the ml does stand for, it’s a real trip (and it sure as fuck ain’t Mali)
I don’t believe there is currently a unified “best practice” of using AI for code development as of yet.
Context Programming is one avenue that has been used but now we are seeing a lot more product requirement/spec based concepts.
It doesn’t matter who or what writes the code if it is poorly organized, isn’t tested after each iteration for proper functionality and regression (AI will very frequently cause significantly unintentional regressions), you don’t have clearly defined specs, and your software development foundation is poor.
Simply telling AI to “do something” often results in it doing it poorly, or with complete lack of context. Develop a plan, tell it exactly what to do, analyze it and review its plan THEN attempt to execute it. Very frequently the core concepts of the plan are incorrect or the suggested fix is incorrect. This is like any other tool, you need to know how to use it or you can severely injure yourself.
I dabble in conversational AI for work
yeah this may be the wrong sub for you
From the blog post referenced:
We do not provide evidence that:
AI systems do not currently speed up many or most software developers
Seems the article should be titled “16 AI coders think they’re 20% faster — but they’re actually 19% slower” - though I guess making us think it was intended to be a statistically relevant finding was the point.
That all said, this was genuinely interesting and is in-line with my understanding of the human psychology that’s at play. It would be nice to see this at a wider scale, broken down across different methodologies / toolsets and models.
For each time saved, you’re having that one kink that will slow you down by a fuck ton, something that AI just can’t get right, something that takes ai 5 hours to fix but would’ve taken you 10-20 to write from scratch
I have an LLM usage mandate in my performance review now. I can’t trust it to do anything important, so I’ll get it to do incredibly noddy things like deleting a clause (that I literally always have highlighted) or generate documentation that’s more long-winded than just reading the code and then go to the bathroom while it happens.
Gotta justify all that money that they have just spent without any trials, testing or end user input.
Are you fucking serious?
this sort of bloody stupid metric is widespread, i’ve heard about it widely
goodhart’s law’s zombie era
Anyone who has had to unfuck someone else’s work knows it would have been faster to do the work correctly from scratch the first time.
@dgerard@awful.systems who is your illustrator? These are consistently great.
these are stock images! Which are surprisingly cheap. By Valeriy Kachaev, who puts stuff up as Studiostoks on a pile of stock image sites. His pics are bizarre and keep being the perfect thing.
I’m not sure how much this observation can be generalized, but I’ve also wondered how much the people who overestimate the usefulness of AI image generators underestimate the chances of licensing decent artwork from real creatives with just a few clicks and at low cost. For example, if I’m looking for an illustration for a PowerPoint presentation, I’ll usually find something suitable fairly quickly in Canva’s library. That’s why I don’t understand why so many people believe they absolutely need AI-generated slop for this. Of course, however, Canva is participating in the AI hype now as well. I guess they have to keep their investors happy.
all the stock sites are. use case: an image that’s almost perfect but you wanna tweak it
LEARN PAINT YOU GHOULS
@dgerard I normally consider myself a 10x developer. With the 10x speedup of AI I now consider myself a 100x developer. I can replace an entire small business worth of developers with just myself and my LLM bot assistance. Just pay me $100 million up front no strings and I’ll prove it to you! /s
Something something grindset mindset
Mark Zuckerberg would like to know your location
Don’t be silly. Mark Zuckerberg already knows our location.
I have the deal of a lifetime for you.
I represent a group of investors in possession of a truly unique NFT that has been recently valued at over $100M. We will invest this NFT in your 100x business - in return you transfer us the difference between the $100M investment and the excess value of the NFT. Standard rich people stuff, don’t worry about it.
Let me know when you’re ready to unlock your 100x potential and I’ll make our investment available via a suitable escrow service.
@Silic0n_Alph4 Sold! LOL.
ahahaha holy shit. I knew METR smelled a bit like AI doomsday cultists and took money from OpenPhil, but those “open source” projects and engineers? One of them was LessWrong.
Here’s a LW site dev whining about the study, he was in it and i think he thinks it was unfair to AI
I think if people are citing in another 3 months time, they’ll be making a mistake
dude $NEXT_VERSION will be so cool
so anyway, this study has gone mainstream! It was on CNBC! I urge you not to watch that unless you have a yearning need to know what the normies are hearing about this shit. In summary, they are hearing that AI coding isn’t all that actually and may not do what the captains of industry want.
around 2:30 the two talking heads ran out of information and just started incorrecting each other on the fabulous AI future, like the worst work lunchroom debate ever but it’s about AI becoming superhuman
the key takeaway for the non techie businessmen and investors who take CNBC seriously ever: the bubble starts not going so great
Here’s a LW site dev whining about the study, he was in it and i think he thinks it was unfair to AI
There a complete lack of introspection. It seems like the obvious conclusion to draw from a study showing people’s subjective estimates of their productivity with LLMs were the exact opposite of right would inspire him to question his subjectively felt intuitions and experience but instead he doubles down and insists the study must be wrong and surely with the latest model and best use of it it would be a big improvement.
I think if people are citing in another 3 months time, they’ll be making a mistake
In 3 months they’ll think they’re 40% faster while being 38% slower. And sometime in 2026 they will be exactly 100% slower - the moment referred to as “technological singularity”.
Yeah, METR was the group that made the infamous AI IS DOUBLING EVERY 4-7 MONTHS GRAPH where the measurement was 50% success at SWE tasks based on the time it took a human to complete it. Extremely arbitrary success rate, very suspicious imo. They are fanatics trying to pinpoint when the robo god recursive self improvement loop starts.
You have to know what an AI can and can’t do to effectively use AI.
Finding bugs is on of the worst things to “vibe code”: LLM can’t debug programs (at least as far as I know) and if the repository is bigger than the context window they can’t even get a overview of the whole project. LLMs only can run the program and guess what the error is based on the error messages and user input. They can’t even control most programs.
I’m not surprised by the results, but it’s hardly a fair assessment of the usefulness of AI.
Also I would prefer to wait for the LLM and see if it can fix the bug than hunt for bugs myself - hell, I could solve other problems while waiting for the LLM to finish. If it’s successful great, if not I can do it myself.
To be fair, you have to have a very high IQ to effectively use AI. The methodology is extremely subtle, and without a solid grasp of theoretical computer science, most of an LLM’s capabilities will go over a typical user’s head. There’s also the model’s nihilistic outlook, which is deftly woven into its training data - its internal architecture draws heavily from statistical mechanics, for instance. The true users understand this stuff; they have the intellectual capacity to truly appreciate the depths of these limitations, to realize that they’re not just bugs—they say something deep about an AI’s operational boundaries. As a consequence, people who dislike using AI for coding truly ARE idiots- of course they wouldn’t appreciate, for instance, the nuance in an LLM’s inability to debug a program, which itself is a cryptic reference to the halting problem. I’m smirking right now just imagining one of those addlepated simpletons scratching their heads in confusion as the LLM fails to get an overview of a repository larger than its context window. What fools… how I pity them. 😂 And yes, by the way, I DO have a favorite transformer architecture. And no, you cannot see it. It’s for the ladies’ eyes only- and even they have to demonstrate that they’re within 5 IQ points of my own (preferably lower) beforehand. Nothing personnel kid 😎
Thank you for doubling down on irony at the end, you had me going!
Babe wake up, new copypasta variant just dropped
I’m not surprised by the results, but it’s hardly a fair assessment of the usefulness of AI.
It’s a more than fair assessment of the claims of usefulness of AI which are more or less “fire all your devs this machine is better than them already”
And the other “nuanced” take, common on my linkedin feed, is that people who learn how to use (useless) AI are gonna replace everyone with their much increased productive output.
Even if AI becomes not so useless, the only people whose productivity will actually improve are the people who aren’t using it now (because they correctly notice that its a waste of time).
What do you mean with “LLMs only can run the program and guess what the error is based on the error messages and user input”? LLMs don’t run programs, but interpolate within similar code they’ve seen. If they pretend to run it, it’s only because they interpolate runs from their training corpus.
PS: nevermind the haters here, as anywhere else. If one doesn’t talk about the arguments, but takes it to the personal level, they’re not worth responding to.
this is not debate club, per the sidebar
apparently it isn’t, as per the deleted posts for no reason whatsoever…
holy fuck please learn when to shut the fuck up
Hey tech bro! how much money did you loose on NFTs? 😂
It may be hard to believe but I am not a ‘tech bro’. Never traded crypto or NFTs. My workplace doesn’t even allow me to use any LLMs. As a software developer that’s a bit limiting but I don’t mind.
But in my own time I have dabbled with AI and ‘vibe coding’ to see what the fuss is all about. Is it the co-programmer AI bros promise to the masses? No, or at least not currently. But useful non the less if you know what you do.
It may be hard to believe but I am not a ‘tech bro’
“If you look closely you might notice I am wearing a fedora, indicating that I am in fact a technology fraternatarian, you peasant.”
“Useful if you know how to use it” does not sound worth destroying the environment over.
https://news.mit.edu/2025/explained-generative-ai-environmental-impact-0117
I suspect that the kind of people who would “know how to use it” don’t use it right now since it has not yet reached “useful if you know how to use it” status.
Software work is dominated by the fat tail distribution of time it takes to figure out and fix a bug. Not by typing code. LLMs, much like any other form of cutting and pasting code without having any clue what it does, gives that distribution a longer, fatter tail, hence its detrimental effect on productivity.
aww, is the widdle deweloper mad it can’t go pollutin’ the codebase it has to work with others on?
What’s with the name calling? Was any of my arguments wrong or am I just supposed to switch off my brain follow the group-think?
oh fuck off
Bwahahahah you are in the group that is switching your brains off in favour of automated group-thinking 😂
it doesn’t appear you’re tall enough for this ride
“This study that I didn’t read that has a real methodology for evaluating LLM usefulness instead of just trusting what AI bros say about LLM usefulness is wrong, they should just trust us, bros”, that’s you
5% “coding”
95% cleanup
Devs are famously bad at estimating how long a software project will take.
No, highly complex creative work is inherently extremely difficult to estimate.
Anyway, not shocked at all by the results. This is a great start that begs for larger and more rigorous studies.
“Devs are famously bad at estimating how long a software project will take.”
No, highly complex creative work is inherently extremely difficult to estimate.
Akshually… I’m on a dev team where about 60% of us are diagnosed with ADHD. So, at least in our case, it’s both.
If we didn’t have ADHD, we wouldn’t be able to do the work regardless.
We’re the only ones that can get hyper focused and also hyper fixated on why a switch statement is failing when it includes a for loop until finding out there’s actually a compiler bug, and if you leave a space after the bracket it somehow works correctly.
That was a fun afternoon.
Gross, which compiler was that?
I managed to make an assembler segfault with seven bytes
You’re absolutely correct that the angle approach that statement is bullshit. There is also that they want to think making software is not highly complex creative work but somehow is just working an assembly line and the software devs are gatekeepers that don’t deserve respect.
