I saw an article in Medium recently which mentions an academic paper on how to improve the context/accuracy from AI on issues involving judgment or uncertainty. The paper recommends to specifically prompt AI to give multiple responses with a probability for each.
For many AI models, this frees the AI from giving the one “most common” answer and instead gives additional background.
I have found this to be very helpful to give me more background on an issuee. I will likely use this technique regularly.
Example here:
https://www.perplexity.ai/search/will-mandamni-or-cuomo-win-the-_kPVY8OQSeahRCFTUt7sZA#0
Do you have a reference or link to the paper?
This is the academic paper:
https://arxiv.org/pdf/2510.01171
This is a less technical summary in Medium:
1 Like
Are you using this type of prompting for certain types of questions or tasks? It seems less useful for questions about facts. I asked Gemini who killed JFK and was disappointed by the probability assigned to the option Lee Harvey Oswald acting alone (30% where the reasonable estimate should be much closer to 100% 
)
I have mostly tried it for questions where I know there is no definitive answer so I am seeking a sampling of responses with the source(s) supporting the various positions. For that purpose it works well.
According to the paper it also works well if you are asking for a creative response, i.e. tell me some jokes or write a poem or limerick.
I am not surprised it is not as helpful on questions that are essentially settled fact.
I’m not sure how to interpret the probabilities. This is a difficult thing, especially when it comes to forecasting, I think. Given how hard reasoning is for LLMs, I’m suspicious.
Thanks for sharing this trick. I will test it and see if it is useful for me.
Yes - incredibly useful.
If I am looking up a really basic fact like the Capital of some state then of course there is only one answer.
But for almost anything I use AI to research, there are multiplke answers and I am basically using AI as an advanced search engine to help me find credible responses on multiple sides of an issue.
1 Like
