Broadly Anthropic’s policy seems to be something like – AI in it’s current form is insufficiently aligned and we don’t have a good enough answer to the alignment problem to proceed safely. Unfortunately we are locked in an arms race with China and if we slow down and China doesn’t then the superhuman AI we’re worried about will come into existence anyways but will instead be representative of the interests of the Chinese government which is a marginally worse world to be in than if such a superhuman AI comes into existence in one of the US companies (which would still be pretty bad).

There’s something very sad about witnessing a mind that has the compulsive need to deconstruct every experience it has, as if it’ll finally reveal the interior and normalcy it so desperately craves. As if through the application of sheer logic and reason it can ferret out its innermost desires and it can finally spend the rest of existence in a state of meditative bliss and complete equanimity with the world. Spoiler alert etc etc

It’s a small computer attached to other small computers, the sum total of humanities inventions since the dawn of time. It is the most salient example of the scientific exercise we as a civilization embarked on. A small compact representation of centuries of scientific progress. “Any sufficiently advanced technology is indistinguishable from magic.” It’s wizardry. It’s something you’d have been crucified for in the 1600s. It has all the information ever written in the world.