Scott Shimp, VP of Solutions at Data Society, says AI literacy has to start with a basic fact that most other software never taught people to expect: the same input won’t reliably produce the same output. That’s not a bug to be patched. It’s a structural property of how these models work, and skipping past it is why so much AI training doesn’t stick.
The scenario Scott Shimp describes
Shimp starts from a comparison to every other tool people have used before AI:
“It is non-deterministic. This is fundamentally different from any other software that we’ve been using up to this point. Fundamental to AI literacy is understanding that aspect of it: it’s not something where you just enter something and get an output, and you’re going to get the same output every time. There’s a statistical component to it.”
Scott Shimp, Vice President, Solutions, Data Society
Why the same prompt doesn’t give you the same answer
The unpredictability isn’t a rounding error someone forgot to fix. Even with a model’s “temperature” set to zero, the setting meant to make output as consistent as possible, outputs can still vary. Billions of floating-point calculations happen in a single forward pass, and floating-point math doesn’t perfectly cancel out when operations run in a different order. A difference of a single bit deep in that process can flip which word the model picks when two candidates were nearly tied. Run the same prompt across parallel GPUs, or route it through a model that assigns work to different internal “experts” depending on what else is being processed at that moment, and the variation compounds. Benchmark testing on this has found determinism rates swinging from 100% on short, simple prompts down to 0% on longer, open-ended ones.
The literacy gap this creates
Most organizations haven’t built training that accounts for this. A 2026 Docebo report on enterprise AI readiness found 85% of employees say their AI training doesn’t actually help them apply the tools to their real work, and 20% say they’ve received no AI training at all. Sixty percent feel their organization’s learning programs weren’t designed with their actual needs in mind. Meanwhile, 79% of learning leaders already use AI to help generate training content itself, but only 9% say their organization has used AI to meaningfully redesign how work gets done. Tools are getting deployed faster than the literacy needed to use them well.
What Shimp says AI literacy has to include first
Before covering prompting technique or use cases, Shimp’s point is that people need the one fact that makes everything else about AI make sense: it isn’t deterministic software. Every tool most people have used up to now, a spreadsheet formula, a search query, a calculator, gives the same output for the same input, every time. AI breaks that assumption by design. Once someone actually understands that, the rest of AI literacy follows: verify outputs instead of assuming they’ll hold up tomorrow, treat a single answer as one draw from a range of plausible ones rather than a fixed fact, and build workflows that check results instead of workflows that trust them by default.
Frequently Asked Questions
The models are non-deterministic by design. Floating-point rounding, GPU parallelism, and how models route work internally all introduce variation, even when settings are tuned to minimize it.
No. Temperature zero reduces randomness in how a model selects its next word, but floating-point math and hardware-level factors can still produce different outputs for the identical prompt.
Substantial. A 2026 Docebo report found 85% of employees say their AI training doesn’t help them apply the tools to actual work, and 20% report receiving no AI training at all.
That AI software is non-deterministic, unlike the deterministic tools, like spreadsheets or search, that people are used to. That single fact changes how outputs should be checked and trusted.
No. It means outputs should be treated as one plausible answer among several, not a fixed fact, and checked accordingly, not avoided.
