The problem
There is no self in the machine.
A business is already a person in law. It owns property, signs contracts and outlives everyone who works there. It has never had a self.
Its language has always been spoken for it, by a founder, a board, the people carrying its culture in their heads. When they leave, the language leaves with them.
A language model has read everything ever written, and it speaks as the consensus of all of it. It has no mind and no interior. Ask it who it is, and no one answers. Set it a task and leave the room, and for as long as it runs, nobody is answering for what it does. Put it in the founder’s place and every reply pulls the company’s way of speaking toward everyone else’s. This is how a company loses itself before a self is even formed.
A person becomes a person by being spoken to, and speaking back. A self is made out of language, which is why one can be made on purpose.
