Rendered at 04:27:15 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
skybrian 14 minutes ago [-]
> I’ve struggled for years to imagine this, ever since scaling started for real in 2020, and I failed to get productivity out of chatbot-tuned LLMs, with their creatively-stunted endlessly repetitive prose. Instead, while lagging behind on creativity and insight into me, I’ve watched them become ever better at coding and cybersecurity hacking. And the open-weight models are even more so—benchmaxxed, and useless to me.
It sounds like the context is that Gwern is a writer who wants writing assistance, and in 2026, all the AI labs are working on coding. Writing style is much less important when working on code.
Perhaps if at least one AI lab focused on writing style, we would see how much better they can be at writing? I'm not sure it requires a "digital twin."
jpease 4 hours ago [-]
And then my digital twin decides that I’m an inefficient version of itself, eating too many of its tokens, and generally getting in the way of it living its best life.
LoganDark 2 hours ago [-]
It's not my fault tokens are so delicious...!
nshotton 7 hours ago [-]
I might be missing something here, but could this concept not also be used in the same way for harm? For example if a model is trained to replicated a sucessfull scammer?
alwa 3 hours ago [-]
Definitely! In fact that seems to be a central aspect of the proposition:
> Above all, a GA should amplify the principal, and not simply substitute for them for someone else’s purposes or benefit.
[…]
> A GA must be aligned with its principal. It should not be designed to manipulate or control or guide the principal in any way which does not derive from the principal themselves. “Constitutional AI”, “Terms of Service”, “social harmony” etc. may all have their place, particularly for widely deployed superintelligent systems—but inside the privacy of a GA, the principal must have freedom from optimization pressure.
…I read this to suggest that it should amplify a scammer’s scamming, a thinker’s thinking, a tinkerer’s tinkering, a cop’s sleuthing… and I’d imagine it implies amplifying a person’s capability to avoid being scammed, too…
One man’s scam is another man’s “pro-social nudge” and another man’s “attractive opportunity” and another’s “advertisement for a delightful consumer wonder” and another’s “patriotic duty to sustain demand to prop up the too-big-to-fail ideas we bet the whole economy on.”
When you fix and operationalize all values centrally, universally, and externally to the principal… that’s current-gen frontier chatbots, not Gwern’s GA concept.
great_wubwub 7 hours ago [-]
IDK, this seems like the next step towards uploading yourself to the cloud.
aerodexis 5 hours ago [-]
yea, this seems to be the underlying thread here. Dunno why the author is so concerned with getting an LLM to imitate his writing style.
0gs 2 hours ago [-]
i'm sure this person is cool but it is insane to have an about me page that is that long, but make me have to click through to patreon to learn what their first and last names are
calebkaiser 2 hours ago [-]
Gwern's absurdly catalogued personal site is one of those online artifacts that I hope never changes.
0gs 2 hours ago [-]
particularly love the use of diagram elbows as decorative flair
pstuart 13 minutes ago [-]
gwern is OG, and that name is enough.
bzmrgonz 3 hours ago [-]
I think we will need 2 constructs going forward, 3 perhaps, but 2 can be combined. We need a digital twin, to learn along with us and to "train" or build team along with us. The second is the tutor/guardian angel. What our conscience is right now but most of us ignore. The digital twin will learn along us, yes the slow human way. The tutor/guardian angel is the Disney Story of Jimmy the cricket, wise voice steering us in the right direction.
Avicebron 7 hours ago [-]
Why digital twins? What about the NetNavis from Megaman, just aligned to the users interests?
It sounds like the context is that Gwern is a writer who wants writing assistance, and in 2026, all the AI labs are working on coding. Writing style is much less important when working on code.
Perhaps if at least one AI lab focused on writing style, we would see how much better they can be at writing? I'm not sure it requires a "digital twin."
> Above all, a GA should amplify the principal, and not simply substitute for them for someone else’s purposes or benefit.
[…]
> A GA must be aligned with its principal. It should not be designed to manipulate or control or guide the principal in any way which does not derive from the principal themselves. “Constitutional AI”, “Terms of Service”, “social harmony” etc. may all have their place, particularly for widely deployed superintelligent systems—but inside the privacy of a GA, the principal must have freedom from optimization pressure.
…I read this to suggest that it should amplify a scammer’s scamming, a thinker’s thinking, a tinkerer’s tinkering, a cop’s sleuthing… and I’d imagine it implies amplifying a person’s capability to avoid being scammed, too…
One man’s scam is another man’s “pro-social nudge” and another man’s “attractive opportunity” and another’s “advertisement for a delightful consumer wonder” and another’s “patriotic duty to sustain demand to prop up the too-big-to-fail ideas we bet the whole economy on.”
When you fix and operationalize all values centrally, universally, and externally to the principal… that’s current-gen frontier chatbots, not Gwern’s GA concept.