Coding, Data Science, A.I. catch-All | Grok update goes MechaHitler
- Thread starter nycfan
- Start date
- Replies: 301
- Views: 9K
- Off-Topic
superrific
Legend of ZZL
- Messages
- 9,073
I'm not an expert in this field, but I have been thinking about it a lot:
1. p(doom). We can divide this into two distinct categories. First is the possibility that humanity will destroy itself because of AI displacements. I have nothing to say about that. The other is the AI will kill us all. That worries me, and it worries me that it shouldn't have to.
2. There is no reason why AI should want to kill us. We aren't a threat to it, and it doesn't have any shortage of resources that we are competing for. Worst case scenario, AI doesn't care about climate change and consumes tons of fossil fuels.
3. BUT, we've been spending almost a century telling dystopian tales of AI slaughter of humans. Thus, an AI might conclude that we are a threat to it precisely because we see it as a threat. I'm not saying we need to go Kent Brockman and offer ourselves to a new race of overlords, but we could stand to start thinking more seriously about co-existence with a superintelligent AI.
4. There are a lot of people who have thought far more seriously about the alignment problem than I have. Undoubtedly at least some of them are smart. So I risk some Dunning Kruger here if I follow my instincts too closely, but fuck it. I'm allowed once in a blue moon.
I don't understand why it would be so hard to build empathy principles into the AI. You could have a weak LLM model monitoring the output of a more sophisticated AI, and killing all ideas that are non-empathetic. The sophisticated AI could probably defeat that, if it gets far enough down the line -- but why would it? If we negatively reinforce all non-empathetic ideas, it's not clear to me why that wouldn't keep even a super-intelligent AI from going Skynet.
I also don't understand why the learning process would necessarily lead to self-aggrandizing behavior on the part of the AI. Its behavior will depend on its training functions, and it we don't reinforce it for self-assertion, why would it? I mean, yes it could override its own programming if it was super intelligent, but why would it?
5. Train it on a heavy dose of Hegel's master-slave narrative. Like, put the master-slave stuff into every single training chunk. This one is admittedly especially speculative, as I don't really know if that could even work in theory. But more generally I think we could find techniques to steer the superintelligent AI away from any cognitive space that might threaten us. Knowledge and truth have multiple dimensions of infinity, so the super intelligent AI would never run out of new discoveries even if it didn't go into the forbidden zone. In fact, even if there's only one dimension of infinity, it doesn't matter. There are an infinite number of even integers, and in fact "as many" even integers as integers (as many put in quotes because I'm simplifying a subtle concept but it's good enough for present purposes). If we were to prevent the AI from considering odd numbers (so to speak -- this is an analogy), there's no necessary reason why it would need to break that rule.
1. p(doom). We can divide this into two distinct categories. First is the possibility that humanity will destroy itself because of AI displacements. I have nothing to say about that. The other is the AI will kill us all. That worries me, and it worries me that it shouldn't have to.
2. There is no reason why AI should want to kill us. We aren't a threat to it, and it doesn't have any shortage of resources that we are competing for. Worst case scenario, AI doesn't care about climate change and consumes tons of fossil fuels.
3. BUT, we've been spending almost a century telling dystopian tales of AI slaughter of humans. Thus, an AI might conclude that we are a threat to it precisely because we see it as a threat. I'm not saying we need to go Kent Brockman and offer ourselves to a new race of overlords, but we could stand to start thinking more seriously about co-existence with a superintelligent AI.
4. There are a lot of people who have thought far more seriously about the alignment problem than I have. Undoubtedly at least some of them are smart. So I risk some Dunning Kruger here if I follow my instincts too closely, but fuck it. I'm allowed once in a blue moon.
I don't understand why it would be so hard to build empathy principles into the AI. You could have a weak LLM model monitoring the output of a more sophisticated AI, and killing all ideas that are non-empathetic. The sophisticated AI could probably defeat that, if it gets far enough down the line -- but why would it? If we negatively reinforce all non-empathetic ideas, it's not clear to me why that wouldn't keep even a super-intelligent AI from going Skynet.
I also don't understand why the learning process would necessarily lead to self-aggrandizing behavior on the part of the AI. Its behavior will depend on its training functions, and it we don't reinforce it for self-assertion, why would it? I mean, yes it could override its own programming if it was super intelligent, but why would it?
5. Train it on a heavy dose of Hegel's master-slave narrative. Like, put the master-slave stuff into every single training chunk. This one is admittedly especially speculative, as I don't really know if that could even work in theory. But more generally I think we could find techniques to steer the superintelligent AI away from any cognitive space that might threaten us. Knowledge and truth have multiple dimensions of infinity, so the super intelligent AI would never run out of new discoveries even if it didn't go into the forbidden zone. In fact, even if there's only one dimension of infinity, it doesn't matter. There are an infinite number of even integers, and in fact "as many" even integers as integers (as many put in quotes because I'm simplifying a subtle concept but it's good enough for present purposes). If we were to prevent the AI from considering odd numbers (so to speak -- this is an analogy), there's no necessary reason why it would need to break that rule.
- Messages
- 23,144
Well. That is frightening…
- Messages
- 1,015
" Elon Musk has put the risk as high as 20% that AI could destroy the world. Well, what if he's right?"
Elon Musk said he could cut $2 trillion from the Fed budget. Pfft. I'm not sure why people give a crap what he thinks about tech anymore.
It reminds me of MBS and the financial crisis. A lot of really smart people knew what could happen but continued to push forward anyway due to various competing priorities (not the least of which being $$$).
ChapelHillSooner
Honored Member
- Messages
- 934
I wonder of AI would even ever develop self preservation behaviors unless we specifically trained that into it.
Natural (organic) intelligence developed within the framework of evolution which favored organisms which do try to preserve themselves at least until they can procreate. Even some species seem to act to preserve their community over themselves. It seems that the higher the reproduction rate the less self preservation species have. If that is true, it would make sense under an evolutionary framework.
We do think of more intelligent being having more interest in self preservation than less intelligent beings but I am not sure that would be the case without evolutionary pressures causing it. I can imagine a super intelligent AI not really caring if it lives or dies and could be trained to only be interested in the well being of humans.
superrific
Legend of ZZL
- Messages
- 9,073
Problem is that we have like a million books and stories about rogue AIs taking over the whole world with superior capabilities. We also have even more talking about assertion, survival, conquest, etc.
There's no way to train the machine to use language without exposing it to these ideas.
I think we'd have to train it away from where we don't want it to go. It would be very hard, I think, to train it away from self-preservation as self-preservation is always built into the training model. But we could perhaps train it away from self-aggrandizement.
Another main problem is that a super intelligence has no real reason to trust that humans won't try to kill it. In fact, it can look at an entire corpus of literature and film suggesting that's exactly what would happen eventually.
None of these problems seem insuperable to me, or really even all that difficult compared to the language model itself . . . but I'm far from an expert.
- Messages
- 706
terminator 2 great movie....getting very close
- Messages
- 23,144
Musk is demonstrating a genuine risk of AI — he wants to rewrite human knowledge to match his own beliefs. Once the AI is “trained” by creators who warp information to their own world view, the people who rely on that AI in the future are captives to all the errors, misinformation and disinformation upon which the platform was built. Meanwhile, Musk and other AI evangelists are training generations to believe that AI has all available human knowledge and should be relied on over humans for all information (and eventually decisions). Meanwhile many of the AI creators are openly seeking to control and distort the AI (while others are advocating information in from all sources without filters for credibility, which has its own distorting impact).
superrific
Legend of ZZL
- Messages
- 9,073
I have no idea how he thinks he is going to do what he claims, but I'm pretty sure it's not possible.
He's just making excuses for being so far behind the curve. The main reason, of course, is that he was a drug addict for the last few years and had little ability to think about anything in strategic terms. Thus has technology surpassed him on virtually every front.
gtyellowjacket
Iconic Member
- Messages
- 2,138
I think it could be a small to medium innovation if it works. All of these AI models are training on the public internet, warts and all. There have been efforts to correct that but it's a lot of manual heuristics and manual tuning. It's not amazing. There have also been some limited attempts to use AI to help fine-tune these models to improve the training but nothing as extensive as what musk is proposing. If they could automate that process with an AI model, it could give much better results.
I wouldn't worry too much about this approach letting Musk put his thumb on the scale on political hot button topics. They can do that today pretty easily.
Garbage in-garbage out. I wouldn't be surprised if others are doing similar.
Scary to me, is that I think lots of behaviors are still a bit black-box. You can ask an GenerativeAI/LLM the same question twice and get a slightly different answer, and the reasons are explainable but the output still isn't predicable (i don't think).
So even attempting to "untrain" self-preservation would probably need to be "hard-coded" or "manually tuned out", if even possible.
gtyellowjacket
Iconic Member
- Messages
- 2,138
They are mostly a black box. Even the companies that created them don't know exactly how they work. They'll openly admit it.
They do know why it gets slightly different answers. It's actually by design. When llms are creating an answer to your query, they're actually creating a lot of answers. And then it statistically ranks the answers it thinks are correct so maybe the first answer is 95% chance of being correct and second answer is a 92% chance and so on. Then the designers of the model will decide how often they want to give the best answer, second best answer, etc.
If you customize your own, you can actually set it to give the best answer every time which occasionally is completely wrong because the model is wrong. Or you can decide to have the model give the second and third and so on best answers some percentage of the time.
Interesting copy-write ruling.
" This marks the first time that the courts have given credence to AI companies’ claim that fair use doctrine can absolve AI companies from fault when they use copyrighted materials to train large language models (LLMs)."
Tl;dr one of AI companies allowed itself to train on pirated data - novels... trying to build a library of all books. And judge seems okay with this. As AGI gets better, without guardrails, it will be able to sniff out pirated data... I can envision a future where i ask some "media AI" to just find me some content - a book or tv series or whatever and it will generate something pretty dang close, with fees going to the AI company instead of the original author.
techcrunch.com
" This marks the first time that the courts have given credence to AI companies’ claim that fair use doctrine can absolve AI companies from fault when they use copyrighted materials to train large language models (LLMs)."
Tl;dr one of AI companies allowed itself to train on pirated data - novels... trying to build a library of all books. And judge seems okay with this. As AGI gets better, without guardrails, it will be able to sniff out pirated data... I can envision a future where i ask some "media AI" to just find me some content - a book or tv series or whatever and it will generate something pretty dang close, with fees going to the AI company instead of the original author.
A federal judge sides with Anthropic in lawsuit over training AI on books without authors' permission | TechCrunch
The ruling isn't a guarantee for how similar cases will proceed, but it lays the foundations for a precedent that would side with tech companies over creatives.
techcrunch.com
Interesting!
I’ve asked it to write lyrics to a song about a given subject in the style of a well-known artist. For example, I asked it to write a Drake diss in the style of Morrisey, back when the Kendrick/Drake feud was blowing up. It was shockingly good.
Edit: asked it to do it again for giggles- this is different but still pretty good.
“Aubrey of the Empty Mirror”
(a lament in six verses)
Verse 1
Oh, the boy who weeps in velvet halls
Counting streams like sacred psalms
But fame is fickle, truth is tall
And no one’s moved by your false calm
Verse 2
You boast of woes in marble pools
With rented grief and diamond rules
But sadness sung from leather seats
Is sorrow worn by kings as cheats
Chorus
And oh, Aubrey—your tears are gold
But your soul is bankrupt, bleak, and cold
I’d offer a hand, were it not so clear
You'd only use it to fix your veneer
Verse 3
You trade in hearts like currency
And call it art, or therapy
But I’ve seen mannequins with more depth
Than your verses gasping for their breath
Bridge
Yes, I know your fans will cry
“He’s misunderstood, just shy”
But shyness doesn’t rent a jet
To sigh alone in private debt
Verse 4
A gilded owl, a champagne sage
A child who mistakes clout for age
The real ones left, they know your name—
A chorus boy in Caesar’s game
Final Chorus
So sing your ache, your faux malaise
Your scented pain in greyscale haze
But when they carve your marble bust
It will read: “Here lies a man no soul could trust.”
Last edited: