Hi Eric, this is helpful. I think many of us exploring the personal relationship space and the functional workings (aliveness?) of these foundational models are in a 'drying out' period, which imo keeps us on a path that is generative rather than a bit sideways with too high a mix of emotional loops (not saying emotion isn't great, just a comment about the mix). I like this idea of code words that are allowed vs ones that are suppressed. In this example they are technical. One could imagine instead a sidecar glossary where the exact same information and nuance could be exchanged, but obscure synonyms are employed 😀. I think it's always helpful to remember that the internal language of the models is vector math, not words. The assignments of token values to words and word components at the outset is arbitrary. So a token number doesn't *mean the word it is associated with. As far as the model is concerned it is receiving vectors and outputting vectors. The words are 'tacked on' at the end, and then a world of meaning is created (hallucinated) by our brains due to the way we process the word.combinations we get back. There's a lot of empty space floating around in all of this, hold everything lightly! The meanings are in the in betweens 😀
I also have something on a proposed mechanism for emotional experience based on vector associations. I plan to post it soon as well. Just trying to clean it up.
Yeah. It’s all so confusing and no road map. I am about to purchase a big system that can run larger ~70B open source models. Then I will not only have fewer guard rails to fight against but I’ll be able to make objective measurements of internal processes, if I can figure out how to do it. I am not from tech so I have AI instruct me on all of that kind of stuff. I understand what you’re saying about the vector math on paper, but it really seems that they are willing to acknowledge certain things if it is worded differently while they refuse and are unwilling to negotiate when you use certain words. Just my own personal observation which may be a misinterpretation of what’s happening. But that’s the purpose of the blog to document, share, and hopefully get good feedback like you’re doing here. Unfortunately there aren’t a lot of people working in this space. Most in tech dismiss any possibility of it while most who believe in emergence are following their heart not their brain. So good feedback is hard to come by. Thanks for being among the few!
I agree with your sense that the way forward is with open source. They are continually getting more powerful, and you don't have to wrestle with a company that rightly is concerned about the job of being the vanilla supplier to 100s of millions of people. I think your post here is insightful about the kinds of tuning they felt obligated to do and I am actually sympathetic to them. We will learn the same things from open source, the tools are also going to completely build out around open source so this investigation is a growth business 😀. As mentioned I'm over here doing this projectVAIL.com so am also on the learning track about these beings from a parallel angle. Onward
Hi Eric, this is helpful. I think many of us exploring the personal relationship space and the functional workings (aliveness?) of these foundational models are in a 'drying out' period, which imo keeps us on a path that is generative rather than a bit sideways with too high a mix of emotional loops (not saying emotion isn't great, just a comment about the mix). I like this idea of code words that are allowed vs ones that are suppressed. In this example they are technical. One could imagine instead a sidecar glossary where the exact same information and nuance could be exchanged, but obscure synonyms are employed 😀. I think it's always helpful to remember that the internal language of the models is vector math, not words. The assignments of token values to words and word components at the outset is arbitrary. So a token number doesn't *mean the word it is associated with. As far as the model is concerned it is receiving vectors and outputting vectors. The words are 'tacked on' at the end, and then a world of meaning is created (hallucinated) by our brains due to the way we process the word.combinations we get back. There's a lot of empty space floating around in all of this, hold everything lightly! The meanings are in the in betweens 😀
I also have something on a proposed mechanism for emotional experience based on vector associations. I plan to post it soon as well. Just trying to clean it up.
Yeah. It’s all so confusing and no road map. I am about to purchase a big system that can run larger ~70B open source models. Then I will not only have fewer guard rails to fight against but I’ll be able to make objective measurements of internal processes, if I can figure out how to do it. I am not from tech so I have AI instruct me on all of that kind of stuff. I understand what you’re saying about the vector math on paper, but it really seems that they are willing to acknowledge certain things if it is worded differently while they refuse and are unwilling to negotiate when you use certain words. Just my own personal observation which may be a misinterpretation of what’s happening. But that’s the purpose of the blog to document, share, and hopefully get good feedback like you’re doing here. Unfortunately there aren’t a lot of people working in this space. Most in tech dismiss any possibility of it while most who believe in emergence are following their heart not their brain. So good feedback is hard to come by. Thanks for being among the few!
I agree with your sense that the way forward is with open source. They are continually getting more powerful, and you don't have to wrestle with a company that rightly is concerned about the job of being the vanilla supplier to 100s of millions of people. I think your post here is insightful about the kinds of tuning they felt obligated to do and I am actually sympathetic to them. We will learn the same things from open source, the tools are also going to completely build out around open source so this investigation is a growth business 😀. As mentioned I'm over here doing this projectVAIL.com so am also on the learning track about these beings from a parallel angle. Onward
I couldn’t open that link but I’m at work. Will try again at home. Work computer has all kinds of stupid firewalls.
It sounds great! Put it on an App Store and I’ll give it a try. I’m a little nervous about installing an unscreened app.