Transformer (machine learning model): Difference between revisions
WikiSysopIG (talk | contribs) |
WikiSysopIG (talk | contribs) |
||
| Line 312: | Line 312: | ||
\rightarrow | \rightarrow | ||
\text{Contextual Representation} | \text{Contextual Representation} | ||
</math> | |||
===Decoder-only models=== | ===Decoder-only models=== | ||
| Line 321: | Line 321: | ||
<math> | <math> | ||
\ | (x_1,\ldots,x_{t-1}) | ||
\rightarrow | \rightarrow | ||
\text{Decoder} | \text{Decoder} | ||
\rightarrow | \rightarrow | ||
\ | P(x_t\mid x_1,\ldots,x_{t-1}) | ||
</math> | |||
Repeating this process allows the model to generate sequences | where <math>x_t</math> represents the predicted next token. | ||
The model generates a sequence by repeatedly predicting the next token based on the tokens that have already been generated. | |||
A simplified generation process is: | |||
<math> | |||
x_1 | |||
\rightarrow | |||
x_2 | |||
\rightarrow | |||
\cdots | |||
\rightarrow | |||
x_t | |||
\rightarrow | |||
x_{t+1} | |||
</math> | |||
Repeating this process allows the model to generate sequences within its supported context and generation limits. | |||
===Encoder–decoder models=== | ===Encoder–decoder models=== | ||