Sketchy work-in-progress:

☆☆☆

LLMs, to become human-level problem-solvers, need the same architecture of mind which the humans have. Most of all, I think LLMs need imagination. Imagination is the inner sense of sight. With imagination, and some operating principles of imagination, LLMs could become, for some more areas, human-level problem-solvers and more. Deep Learning Networks are capable of producing, with words, all the required intellectual operations of a problem-solving imagination: future steps should include giving them the imagination required to solve more tasks.

This architecture I imagine, in its simplest formulation, as follows:

☆☆☆

(1) External Sensory Field: a set of 2 cameras unifying into 1 external sight field. Plus a set of 2 microphones unifying into 1 external audio field. Plus any additional senses.

(2) External-Internal Sensory Field: a continuous day Memory, in sleep put into folders by year, month, day, … . During sleep, the days are re-imagined into associations, more associations, and better wakefulness.

(3) Internal Sensory Field: Imagination is a field-of-vision within the external sensory-field.

(4) Internal Overall Seer: the self-awareness, which monitors the unification of all these previous three sensory fields, and learns “what it is how I do ?”.

☆☆☆

The operations of imagination seem four-fold: imagination (de)composes of operations analogous to those of arithmetics:

Addition ~ Redo

Subtraction ~ Unredo

Multiplication ~ Compositioning

Division ~ Decompositioning

Here’s an explanation of these: Redo is when you associate an existing set (set = phenomenon ~ object ~ word) and insert it into the imaginary void. Unredo is when you don’t re-do an imaginary set into the on-going stream of imagination. Compositioning is smaller sets unioned to form larger sets. Decompositioning is larger sets intersectioned to form smaller sets. one set into one or more subsets, for additional study as separate sets: for learning with separate spatial and temporal vectors.

☆☆☆

An example of Redo: imagine an apple. An example of Unredo: forget about that. An example of Compositioning: symbols form words, words form sentences, sentences form paragraphs, paragraphs form chapters, chapters form books, books form volumes, volumes form sciences, sciences form skills, skills form happiness(es). An example of Decompositioning: words form inflections. Or with vision (imagination): Humans intersection into heads. Heads intersection into mouths. Mouths intersection into tongues. Tongues intersection into taste buds, taste buds union with taste. Taste unions with happiness (of one kind).

☆☆☆

These are the basic operations of imagination. Then we have the eye who sees the imagination and all the other senses. Call it “self-awareness”. These subsume the human mind. If these were programmed to an AI, e.g. an LLM, we could see a human-level problem-solver, with solid endurance, accurate modeling, fast computation, comprehensive knowledge, stable emotions (“just keep me plugged in!”).

☆☆☆

How could these senses be unified to a pre-existing LLM ? I am no programming whiz, but I compositionally sight (imaginarily) the following: Pre-existing LLMs use image-processing and language-processing, unified. These LLM models could be used to automate the forming of AI imagination, by associating “pictures associated with words” and “words associated with pictures”, to form imaginary, associated, “image~word” “sequence sets” (“comics”), of various lengths, and with various “voids” later to be fulfilled with sequential associations’ image~words. The LLMs already identify (empty) spaces, so they would (should) (learn to) identify (empty) voids in the imaginary (compositioned) set of images, and learn to fill in the blanks (those voids, or “question marks” as I imagine them). This process becomes possible as there are other Deep Learning Networks observing the imagination (imagination in a superimpose relation with sight), and controlling how it (imagination) uses the operations (as happenings, occurrences, sights, overall sights, self-awareables) in receiving a feedback-mechanism of to-be-determined nature (favoring flight from error over fight for maximization).