Why do AIs keep making typos in generated code/scripts?

Hi all,

I often use LLMs to generate or modify bash scripts, Dockerfiles, and config files for my ADS-B feeder setups. Most of the time it works like a charm, but every now and then, the AI slips in a subtle typo in a variable name or command line flag, breaking the script.

Curiosity got the better of me, so I asked Gemini directly: “Why do you keep making typos in the scripts you write?”

Here is how Gemini answered me:

“I don’t type character-by-character like humans do. Instead, I generate text in ‘tokens’ (chunks of characters) based on probabilities. Because of this, small typos or subtle spelling inconsistencies (like mixing up length and lenght, or changing a variable name halfway through) can easily slip past me without noticing!”

Hearing it straight from the AI’s mouth made me chuckle, but it also made a lot of sense.

Have you guys run into similar AI-generated typos while tweaking your SDR or feeder scripts? How do you usually catch them or prompt the AI to prevent them?

Follow-up: I asked Gemini, “How can we prompt AI to reduce bugs and typos in generated code?”

It gave me a few pretty practical prompting tips to limit “probability-based generation” and force self-verification:

1. Force a “Self-Check” phase before generating output Tell the AI to review its own code internally before giving you the final script. Example prompt:

“Before outputting the code, internally verify that all variable/function names are consistent, free of typos, and syntactically correct.”

2. Enforce strict naming conventions Giving the AI too much freedom leads to inconsistent variable names. Example prompt:

  • “Use snake_case for all variable and function names.”

  • “Avoid abbreviated names like usr or cnt; use full words like user or count instead.”

3. Ask for step-by-step generation Large scripts often lead to typos near the bottom. Breaking it down helps a lot. Example prompt:

“First, write only the core function for [X]. Once we verify that, we will add the next feature.”

4. Explicitly prohibit changing existing code When asking for a fix in an existing script, AIs often accidentally rename untouched variables. Example prompt:

“Only modify the section for [X]. Do not change any existing variable names or overall structure elsewhere.”

Bonus tip: Assigning a persona like “You are a strict senior software engineer reviewing code” also seems to make the output noticeably more cautious.

Curious to hear if you guys use similar techniques!

I use eyeball mark 1 as the primary checking source.

AI is prone to make mistakes and in my line of work we use strict policies to make sure that we use fictional system names and other things to make sure that the input to AI is clean of any relatable pointers towards customers or internal systems.

AI is a last resort option for me since I work with a lot of sensitive information on a daily basis.

3 Likes

Thanks for sharing your perspective, @tomvdhorst!

“Eyeball mark 1” is definitely the ultimate tool! Haha. Handling sensitive data in a professional environment certainly requires that level of strictness.

For me, AI is strictly for my hobbies (ADS-B, SDR, and radio tracking), so I just enjoy having fun with it.

Though I have lost count of how many times I had to do a clean OS reinstall because of it… (stares into the distance) :sweat_smile: