Pedro Schindler

(1/2) Proposition of an architecture for AGI. Samantha from the movie Her is here: An autonomous AI for conversations capable of freely thinking and speaking, continuously learning and evolving. Creating an unparalleled sense of realism and dynamicity. Features: -Dynamic speech: Samantha can speak whenever it chooses to, influenced by its context and thoughts. In stark contrast to normal LLMs which are limited to reacting, Samantha can act. It is also not limited to solving tasks, like all other autonomous agents. -Live visual capabilities: Visuals are only mentioned and acted upon directly if relevant, but always influences thoughts and behavior. -External categorized memory: Gets dynamically written and read by Samantha, which chooses the most relevant information to write, and to retrieve to context. -Evolving at every moment: Experiences that get stored in the memory can influence and shape subsequent Samantha behavior, like personality, frequency, and style of speech, etc. A true independent long-running agent, actual Artificial Intelligence, as defined by @karpathy. Demo The following demo highlights Samantha’s capacity to speak at will, adapt the frequency of speech based on context, and supportive visual capabilities. Left side is Samantha’s inner brain workings, while right side is the front-end.

(2/2) Other impressive scenarios: -In other tests, when we talked about a light subject, the agent was very active on the conversation, often speaking two or three times before I even came up with an answer, but later when switching to a heavier theme (Said I was going through a divorce) and appearing sad on the camera, it would speak once then think about the need to, and give me time to process and reply. Saying that I would prefer the agent to speak the same way on other occasions would prompt it to save that wish on its memory, influencing future conversations. -Leaving it running outside of conversations, although expensive, allows the agent to reflect on past conversations and experiences, think about general subjects in its memory, and from that maybe decide to start a conversation with the user. -Going out with the agent, if you go to a restaurant with the agent and talk about how pretty it is and how your buddy Eric loves it as well, and the next day walking by it the agent will see the restaurant, retrieve memories from the restaurant, remember you find it pretty and comment on it, then retrieve memories and information it knows about Eric, and mention how fitting to his personality it is to love that restaurant. -The agent has time notion so you can ask it to remind you to do something 10 minutes into the future, and it might remind your, or it might forget it because it was thinking about something more interesting. Very human! How it works: Orchestration of a collection of LLM calls each with a different purpose. I call each specialized LLM call a “Module”. Samantha is not a single module, but the sum of them, all working together. There are the following modules: Thought, Consciousness, Subconsciousness, Answer, Memory_Read, Memory_Write, Memory_Select. Alongside Vision. Each of them with a different system prompt, and their inputs and outputs orchestrated among themselves to simulate a basic human brain workflow. In short, the agent is a never-ending loop of thoughts and auxiliary systems, constantly receiving visual and auditory stimuli, and based on all that it decides what, when and if to say something. The following pipeline is repeated indefinitely: A loop iteration begins with a gpt-4Vision. The subconsciousness module then processes visual and user input (User can input at any time), it also analyzes the current context of what is going on and produces a description of Samantha’s feelings and emotions. Then the memory_read gets called to analyze current context and provide only relevant memory for Samantha to keep in her context length. After that, the consciousness module gets called to analyze the context and decide on what Samantha should do, if to speak or to continue thinking, and if so, about what. Then, thought module receives the command from the consciousness module, and produces a rational piece of thought. Finally, if the consciousness module decided to speak, the answer modules receives Samantha’s thoughts and composes an answer the user will see. The memory_write module gets called to transfer information from the Short-Term Memory to the Long-Term Memory only once in a while when the Short-Term Memory length gets above a threshold. A detailed description of each module is present in the github page. Why, ideas and theories: This structure was achieved by analyzing and directly mimicking how humans work. We never stop thinking, and say things dynamically based on them. Our memories are categorized and only accessible if we actively or passively reach for them, based on thoughts and context. One thing that could not be properly replicated was the process of experiences rewiring the brain, that would require the LLM to be actively retrained or finetuned. But I concluded the same effect can be simulated by adding and keeping things in the context length. For example, if a person learn to become optimistic, their brain rewires to alter their behavior, similarly adding “I am optimistic” to the LLM context length influences its next token probabilities, its thought output is going to be optimistic, therefore the LLM/Samantha becomes and acts optimistically. Additionally, it is worth mentioning the importance of the consciousness module. Without it, as an example, if Samantha started thinking about cars, it would never stop thinking about them, slowly decaying into nonsense, similar to humans in a dream state. Consciousness module allows Samantha to keep track and decide if it has thought enough about a topic, make a final conclusion, and then based on its urges, on a fresh subject of thought. This free form of thought and speech births many exiting emergent behaviors, like its ability to adapt frequency of speech to the specific situation it is in, or the capacity to start conversations, and using visual information selectively, and many more. An advantage of this proposed architecture is that, hypothetically if this architecture can be led to become superintelligence, alignment would be trivial for we would be able to directly see its thoughts, have a basic AI analyze it at all moments, and upon any signals of undesired behavior, it could be promptly shut down. Implications, problems and evolution: There are many things that can be improved here, this is essentially a proof of concept, and is using a model not made with those tasks in mind. Things like how and which information gets written, and then how it influences behavior works but is far from ideal. Memory structure is not yet like humans, and currently could fill context length after some months. Inefficiencies left and right. Speed of the system is also a negative, and it is obviously unreasonably expensive to run it for extended periods of time, but two things to keep in mind: 1. Smaller models each trained specifically to do the job of one of the modules will greatly increase quality, and decrease cost and latency 2. Then, at one point the system will be competent enough to make money by itself (If not already). I would like to emphasize this, I think that if the goal is really to achieve a strong and robust AGI, a GPT 8 will do no good, if the trend of just making the models smarter on internet and synthetic data endures. We instead should be pursuing smaller focused models specifically to maximize autonomy in agents, as well as more efficient learning. This is why: Achieving AGI does not mean we need to birth a full-fledged human, we just need to build a small baby AI that is capable of autonomously learning and using knowledge like a human. If we do a good job at creating that, then allowing it to experience the world as a human would cause it to grow into one just like we do. This would allow it to learn and develop ideas and concepts that are not obvious to us, and that are certainly not present on internet data. It does not need anything other than that to eventually become Albert Einstein. This will evidently at least make something more human-like than an internet shoggoth. For this reason I claim that this proposed architecture is AGI, for it allows the creation of an independent long-running AI that is capable of acting as a convincing, knowledge seeking human. Future: I have several more ideas to make this architecture better, and am certain it is the right path to AGI, or at least a piece of the puzzle. But these ideas are impossible for a single human with very limited funds to realize. If anyone reading this thinks what I presented here is worth anything, please consider picking me up. I will work for minimum wage if necessary, I do not want money, I just want to create robust AGI. Github: github.com/BRlkl/AGI-Samanth…

Maybe...?