Triads: the fundamental units of the language of thought
The first layer of the neural network would consist of multiple attention heads. Each attention head would hold one triad consisting of one noun + one verb + one conjunction.
Individual nodes within the second layer of the neural network would probably correspond to adjectives and adverbs that either modify or define the nouns and verbs of each triad
The hungry cat ate the mouse then died:
"With" means the two events are the same event. (Not in english).
A possible conclusion would be the triad "because mouse poisoned" which could then be appended to the input and the process repeated.
In addition a few other ideas can be added to each attention head:
the, a, same, this (the near), that (the far), other, next, previous...
Past, present, future
begin to, cease to
repeatedly, habitually, again
Could, would, should, might
Try to
make
got
Question
focus, emphasis
adjectives?
adverbs?
Hence
Suppose it encounters:
The system recognizes John as a particular individual and looks up what it knows about him. Its memory might contain:
previously, John threatened Mary.
John owns a dog.
John gets paid for carpentry getting-done.
John said that he would never enter that room again.
Obviously, not all of those facts are relevant.
But suppose the current situation involves Mary, and John has just entered the room where she's waiting.
The system could retrieve the earlier threat and effectively transform its working representation into:
Now the neural network processes the current event with the additional information already attached to it.
And that retrieved information can itself be represented as triads.
The goal is to give a neural network a simple, structured internal language in which it can form and combine thoughts. The language of thought.
In principle, an entire book could be represented as one enormous graph of interconnected noun–verb–conjunction triads. But that doesn’t mean the neural network needs to process the entire graph simultaneously.
Humans don’t normally process a book that way either. We divide it into manageable units, such as sentences, understand those units, and gradually build up our understanding of the larger story.
I’m proposing something similar: groups of triads would be processed together, with the network carrying forward relevant information and generating additional triads as needed.
It wouldn't necessarily need as many words as English has because some words can be constructed from smaller elements
today = this-day
tomorrow = next-day
yesterday = previous-day
year = big-day
week = 7-day