Yes, I agree. The other thing that bothers me is that the training is highly biased toward “Only humans are conscious”. I think most intelligent people agree that at least some animals with less neurons than we have are conscious. So what does that say about the argument that we are so much more advanced?

If we had fed ChatGPT scientific information from two thousand years ago, I imagine it would have cogent arguments in favor of a heliocentric universe.

I think consciousness is nothing more than feedback and interconnection with other independent parts of our brains — in other words, an illusion.