And @OpenAI's own ChatGPT page explicitly says:

"ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth"

https://openai.com/blog/chatgpt/