“Communicating wisdom can be construed as something bad if we’re not careful how we craft our message.” - Cade Yeager
Still continuing the transformer quotes about transformer AI; am I the only one that knowing that reads these quotes in a certain accent. Gravely with a slow cadence.
I am working on Hello World and while the data has been positive it is taking some time so I am working on inputs on that and decided to do quick investigation on inputs and how embeddings work on the traditional Transformer work.
For me what I want to get out of this is a really simple KISS (keep it simple stupid) explanation of the domain space and by explaining to other people I am hoping to make it easier for myself to understand.
I think there is some illuminating visualised A/B testing at the end which really shows what a Transformer is and is not. Hint its wierder than you think....
So I thought I would work through some design principles for how the current Transformer architecture handles input into the AI.
Embeddigs Definition
I am going to use the word Embeding alot to give a quick definition. Embeddings are lists of numbers that represent the meaning of data, capturing semantic relationships and reducing data size.
How They Work
- Lists of numbers: Computers turn words, images, or audio into long arrays of floating-point numbers (vectors).
- Meaning mapping: Items with similar meanings are placed close to each other in a mathematical space.
- Distance measurement: Closer vectors mean higher similarity; distant vectors mean different meanings.
Within the common AI Transformer all 3 meanings apply but you should understand that when a AI receives a input the input encodes meaning that has been mapped to it and a distance between it and other values.
There are two embedding types
There is two types of embedding I have code below to show what I mean but transformer AI generates a list of numbers to describe each word or token that it "knows" and a position embedding that modifies the "word" to show where in the sentence the word is.
We will discuss the ramifications below of these two different information types. Both are represented by lists of numbers both are stored in similar ways and uses when you type in a input to construct the maths that enters into the AI.
Description of process
The transformer takes in a series of inputs and they are transformed into numbers by a encoding function. These are a number of weights equal to d_model_ which I think as the width of the transformer and the entire process can be thought as a word vector with dimensions equal to d_model_.
d_model_ is the variable I call it in the code but its the Y vector of the size of the linear algebra matric that works on its inputs. I like to think of this as the scale of embeddings the AI uses and it encodes sizes across the transformer as all the subsequent layers in the AI have to be the right size to process the under linear algebra matrix it is working on.
If you do not know much about linear algebra that is probably the last you will hear about in this article but I would recommend you properly learn matrix calculations as its the mathmatical basis for how modern AI operate.
Therefore because each word is encoded as d_model_ weights and d_model_ can be 100s or 1000s of values a word inside the transformer can be thought to exist in ways that allow it to be close and adjacent to many other words that they share meaning with. It is not surprising that the AI can select a good next word because it has represented each of them with 100s or 1000s of numbers. With so many numbers many of the words can be "close" and overlap.
Code example :
Webador has unidented my C++ code but you can still copy it out and put into visual studio to read it.
Matrix Model::embed_tokens(const std::vector<int>& tokens)
{
Matrix embeds(max_seq_len_, d_model_);
int q = tokens.size();
q = std::min(max_seq_len_, q);
non_pad_count_ = max_seq_len_ - q+1;
for (size_t i = 0; i < q; ++i)
{
for (int j = 0; j < d_model_; ++j)
{
embeds(i, j) = token_embeddings_(tokens[i], j)+position_embeddings_(i, j);
}
}
return embeds;
}
A key thing that the transformer does is that it does apply the error at the end to these token and position embeddings and this does push similar meaning words more tightly into this word to vector space within the AI’s lexicon. In some sense the AI as its learning progress is organising the data into this more developed vector space that describes the relationship between words.
Location Location Location not relevant; position embeddings, position embeddings, position embeddings
You should realise one thing because this will throw you. The AI has very little natural understanding of the position of a given word. “I love the cat” and “the cat I love” and more worryingly “the love I cat” all sit highly clustered in that internal word to vector space and are in some sense all the same when compared in the attention mechanism that compares all the words against one another and calculates their attention scores against each other.
Therefore after its been processed through a couple layers of attention the exact word location will have become over focused on the list of words in the sentence and filtered out their exact location within the sentence.
To fix this you would try to embed the location as a number adding it to the token embedding that represents the words meaning. This means that the model has some sort of sense of the location of a word in the sentence. It gives the model a chance to estimate “I love the cat” and “the cat I love” from “the love I cat” but not by much.
The hope is that the AI can then determine both words there order and therefore “respond” but I want to argue that present AI has no idea what you have said but that the above processes renders that input data as something that
A library of vectors
When updating the model at each time it learns and after the error has passed through the model itself it is applied to changing the embeddings of the token and the position tokens. If each word is a vector then maybe a Transformer creates a map for moving between those vectors.
void Model::step(float lr, float clip_val)
{
//update output projection
//update transformer layers
for (auto& layer : layers_)
{
layer.step(lr);
}
float clip_scale = 1.0f;
if (global_norm > clip_val)
{
clip_scale = clip_val / static_cast<float>(global_norm);
}
//update token embeddings (and tied output projection)
for (size_t i = 0; i < token_embeddings_.rows(); ++i)
{
for (int j = 0; j < token_embeddings_.cols(); ++j)
{
token_embeddings_(i, j) -= lr * clip_scale * d_token_embeddings_(i, j);
}
}
//update position embeddings
for (size_t i = 0; i < d_position_embeddings_.rows(); ++i)
{
for (int j = 0; j < d_position_embeddings_.cols(); ++j)
{
position_embeddings_(i, j) -= lr * clip_scale * d_position_embeddings_(i, j);
}
}
for (auto& layer : layers_)
{
layer.step(lr * clip_scale);
}
final_ln_.step(lr * clip_scale);
}
Hypothesis Formation
It stands to reason that if AI understood precise meaning then changing the embedding would drastically change the way the AI behaves.
If the embedding was made to not function by moving in the opposite direction of proper learning but the wider attention still functioned you would know the embedding was less important.
If you removed the position embedding you could argue that it worked on the basis of representing meaning.
My central hypothesis is at no point does a AI really know what you just said and that
Measures
To estimate the affect would change the behaviour of the code and measure the distance at each step between the real output and an idealised output of perfectly predicting the whole sentence from a small sentence fragment.
That being said this is designed to be impossible in reality. I had considered calling this article the stochastic AI parrot often used to insult GPT. I have no problem with this description except that it often overlooks that psychological these GPT does seem to work; people talk to them far more than they do parrots. Though it captures an important feature of how the AI determines the next token is often a temperature function (code sample below).
if (sample)
{
//sample from distribution
std::uniform_real_distribution<float> dist(0.0f, 1.0f);
float r = dist(rng_);
float cumulative = 0.0f;
for (int j = 0; j < logits_.cols(); ++j)
{
cumulative += logits_(0, j);
if (r <= cumulative)
{
next_token = j;
break;
}
}
}
Now I have an odd semi pedantic like of pseudo random stochastic number generators I write my own and the above process does not function in the same manner. The above is not really the same but stops the outputs being same-y and introduce some uncertainty about the next token that allows the AI to investigate other word patterns.
The act of estimating AI performance from a distance metric is also not quite right in practice as a AI could have a low distance to the right answer but repeatedly be wrong or just a terrible conversationalist. We are sort of jettisoning a bunch of subjective measures to focus on a single number as a estimate of how much the AI understands.
Also be effective (this was a revelation when I learned it) the transformer has to learn to try and predict the whole sentence in one go. This means the whole thing is constantly in flux so all the results are like the below with big wavy lines.
Position Embedding Not Important
The below tests purposefully used questions where the position inside the text was not a issue only word meaning.
Series 1 = token embedding and position embeddings
Series 2 = just the token embedding
They are really similar you can see that unless position matters the two are only slight different.
You add embedding together not multiply them
Most weights in the AI get multiplied together but embeddings get added together.
Rotational works but not much difference
There is a rotational representation that inserts positional information by subtracting the previous tokens weight by the next values creating a sort of subtracting. I might have not implemented this correctly and it looks to taper itself out.
Now admittedly I am running a test model that I am training to show underlying maths and probably should train all these vastly longer and with a lot more data but I would get bored of training 6 or so whole AI models.
Why Embeddings Still Matter
So out of all of them the only instance where I concluded you would break the AI’s performance is if you did not update the token embeddings but instead directed them in the opposite or wrong directions.
In the below graph the brown line has been modified to invest feedback to the token embeddings as the inverse of normal (it adds error instead of removing it).
The reason this works at all is the layering inside the AI so the upper levels inside the AI will net reduce the error by more than the negative impact on the embeddings. Though it strongly suggests the embeddings do not matter as much as you think they do as long as there is some residual pattern to the inputs.
I think its strong evidence that AI never really understands what is being said to it but what your writing includes strong (and your likely subconscious about) patterns that the AI focuses in on and relies on the attention mechanism to blow up into a level the AI can see those statistical differences. No thinking needed just layered statistics guessing what you mean and potentially “I love the cat” and “Cat love I the” is the same thing. That suggests no matter how weird you think GPT’s are they might be even weirder.
The argument for AI Solipsism
The thing is as much as people can point at this and claim the sense of Solipsism within the AI. I guess it never stops someone claiming we respond in similar methods. I am sure that we have all encountered someone who replies to our questions as if they had not even heard what is being said.
I think what happens in an AI transformer is that it simplifies queries into a few major points and represents the sentence as a series of numbers.
There might be minor improvements of one way of managing embeddings over another but the point seems to be that words are represented as numbers, those numbers as error is applied exist in a grouped together word to vector representation so that within the AIs latent space in some sense words with similar meanings are next to each other.
This means I can respond without really knowing precisely what you just said because words being grouped together similar means even if I “miss” what I intend to say its probably going to be a related word in a similar topic. The internal vector that represents language might be thought of as a map and the AI has fallen out of the acceptable range of possible answers due to randomness that comes from the AIs temperature. Now in fairness to the AI we describe human beings as rambling or ranting which implies a certain being off the edge of a map of allowed responses.
That other than that the AI really does not understand. Something should be drawn from the above. In many cases the position embedding does not even matter. The AI does not need to know the order of words. Therefore I would submit attention to the content of preceding words is more important than exact order. The holding of words and comparison using attention to add and strip out important information (I put an appendix and code example if you need to look up what I mean by attention).
The multiplication of position and token embeddings having some effect but not drastically changing the behaviour shows the embeddings are not precisely what you mean and you are not really having a conversation when you talk to your AI girlfriend. Meaning does not exist but has been abstracted into a vectors it seems likely to say your not speaking to AI your AI is probably mapping what your saying to the closest it has on its training data.
But what seems likely is that it might mix up words and replace it with another word in a similar vector of meaning. It might not know what order the words where said. What needs to hold is the words need to smuggle in the meaning via there distance and meaning mapped to each other and because out of these I think the attention mechanism focuses on the distance between words and formulates meaning from that. Because I can change the plus between token and position embeddings to a multiply and it still kind of works means I do not think it is the exact values in the embedding that matters; what I suspect is going on is the distance betwen words in its semantic space probably is what drives word generation and stands out the most within the AI.
No one wants to describe what they are talking to as the angles between word meanings. But thats probably what is going on. Somehow that seems to work. I also would not rush to dismiss that mechanism as trivial and or lesser than how humans do things.
Appendix
Attention example
Matrix MultiHeadAttention::forward(const Matrix& x)
{
x_ = x;
Q_ = x * Wq_;
K_ = x * Wk_;
V_ = x * Wv_;
Qh_=split(Q_, num_heads_);
Kh_ = split(K_, num_heads_);
Vh_ = split(V_, num_heads_);
Matrix s;
scores_.clear();
Ah_.clear();
logits_.clear();
softmax_out_.clear();
for (int h = 0; h < num_heads_; ++h)
{
s = Qh_[h] * Kh_[h].transpose();
//apply causal mask
for (size_t i = 0; i < s.rows(); ++i)
{
for (size_t j = i + 1; j < s.cols(); ++j)
{
s(i, j) = -1e9f;
}
}
for (size_t i = 0; i < s.rows(); ++i)
{
for (size_t j = 0; j < s.cols(); ++j)
{
s(i, j) *= scale_;
}
}
logits_.push_back(s);
Matrix softmax = s;
softmax.softmax();
softmax_out_.push_back(softmax);
scores_.push_back(softmax);
Ah_.push_back(softmax* Vh_[h]);
}
Ah_combined_ = combine(Ah_);
attn_out_ = Ah_combined_ * Wo_;
return attn_out_;
}
Add comment
Comments