"This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating"

What? These models are all trained from books and text that are scraped from the internet. ChatGPT literally used reddit in its training data afaik.