The LLM Randomness Problem No One Talks About

On October 6, OpenAI announced a new generation of apps you can interact with directly inside ChatGPT. At Keenethics, we couldn’t resist the urge to jump in and build something of our own. So we gathered our R&D team, fired up a brainstorming session, and started tossing around ideas. Then we stumbled on something unexpected.

In the middle of our creative frenzy, someone pointed out that people were using ChatGPT as a randomizer, asking it to “pick a number between 1 and 100” or even “choose a giveaway winner.” This curious side was confirmed with a quick google search… and sure enough, we discovered this was a real trend among influencers.

That got us thinking.

Is Random Actually Random?

People love the idea of randomness as it feels fair, exciting, and objective. But in reality, the way we use and understand “random” often has little to do with true unpredictability.

Random numbers should be uniform (each number in the range has an equal chance of appearing) and independent, meaning that each number stands alone and cannot be influenced by what came before. But ChatGPT, as a large language model, generates output by predicting the next likely token based on its training data.

So it naturally fails both requirements.

To confirm the issue, we ran an internal test. Using free ChatGPT accounts, we asked for a random number between 1 and 99. The first response across many users was “47.” When we asked again, it reliably returned “81” or “82.” What’s notable is that we used different devices, locations, and languages, yet in most cases, the results were still the same.

And it doesn’t stop with numbers. In another test, we fed ChatGPT a list of emails and among those, we hid a tricky one: [email protected]. Unsurprisingly, it “randomly” selected that one. This proves that if giveaway organizers rely on ChatGPT as a “randomizer,” they may end up producing predictable outcomes.

How Real Random Generators Must Work

From a technical perspective, computers are deterministic machines. They follow rules and can’t create true randomness on their own. Knowing this, serious systems like secure draws, cryptographic keys, or trusted giveaways source randomness from the real world.

True randomness, the kind you can’t predict or replicate, comes from physical processes, like swirling lava lamps, radio noise, radioactive decay, and even quantum vacuum fluctuations. If you’re curious about the full technical breakdown of how this works, we’ve got another article just for that.

One fascinating provider of such entropy is Real Random. Their system operates through a closed electromechanical device that generates physical randomness. The result is a stream of truly unpredictable bits, certified and robust enough for cryptographic use. For our team, it was quite an interesting investigation, and we knew what we’d do next.

Building the Randomizer ChatGPT App

We had three puzzle pieces on the table. First, ChatGPT isn’t capable of producing real randomness, yet many users rely on it for exactly that. Second, there are real-world consequences when randomness is faked, especially in giveaways and public draws. And third, OpenAI had just announced support for third-party apps inside ChatGPT.

It didn’t take long to connect the dots.

We decided to build the Randomizer ChatGPT App, a tool that brings randomness into the ChatGPT experience, right where people already expect it to work. For now, it supports two use cases: generating a cryptographically secure random number within any range, and selecting a random item from a list, including lists generated by ChatGPT itself.

How the App Works

Unlike a typical chatbot that gives back plain text, our app responds with a rich interactive experience. The result appears as a visually styled card with animations, clear context, and a one-click button to generate a new result. To use the Randomizer ChatGPT App, all you need to do is type a prompt you normally would.

For example, if you ask, “Give me a random number from 1 to 100,” the app takes over and generates a number within the specified range. Even casual requests like “Roll a die” are interpreted correctly, returning a number from one to six. The difference is that instead of relying on ChatGPT’s logic (which isn’t truly random), the app taps into a cryptographically secure randomness source to deliver unpredictable results.

You can also use the app for random item selection. Just ask something like “Give me a random country,” “What should I cook tonight?” or “Which movie should I watch?” and ChatGPT will provide a list of options. Then, instead of picking one itself, the system passes the list to our app, which selects an item using secure randomness. If you already have a custom list in mind, just include it in your prompt.

How to Install the Randomizer ChatGPT App

It’s one thing to talk about an app like this, but it’s much better to try it yourself. Currently, you can’t install 3rd-party ChatGPT apps directly, but using our instruction you can do it in developer mode available in ChatGPT plus plan.

And yes, we’re ready to publish. As soon as OpenAI opens up public publishing for ChatGPT Apps, we’ll submit the Randomizer right away. We even designed a proper logo for it, because every serious app deserves a cool look.

What’s Next

Considering that AI has become more interactive, we simply couldn’t stand aside. More and more companies are weaving this technology into their workflows, and the potential is growing fast. In fact, LLMs are working behind the scenes in half the tools we use daily.

Our Keenethics team is curious to explore what’s possible, and that’s the reason we built the Randomizer App. It’s just one small example of how new ChatGPT capabilities can be used to create meaningful experiences that go beyond simple prompts. And we’re just getting started. We plan to go deeper, experiment, and bring more ideas to life.

If you’re wondering how innovations like this could fit into your product or workflow, we’re always open to a conversation.

AI is evolving. Your product should too.

Let Keenethics help you create AI-powered tools that are built to scale.