|
<< Click to Display Table of Contents >> Navigation: 3. Script Language > AI - Artificial Intelligence Commands > AIC. - Artificial Intelligence Command > AI - Set Parameters > AIC. - Artificial Intelligence Command |
MiniRobotLanguage (MRL)
AIC.Set Top_P
Set the Top_P Value in the LLM
Intention
The `top_p` parameter, also known as nucleus sampling or `p`-sampling, is another way to introduce randomness into the model's responses. Instead of selecting from the top `k` most likely next words, `top_p` sampling chooses from the smallest possible set of words whose cumulative probability exceeds the `p` threshold.
The `top_p` value is a float between 0 and 1.
- A `top_p` value of 0.0 would make the model deterministic, always choosing the single most likely next word.
- A `top_p` value of 1.0 means the model considers all possible next words, leading to maximum diversity in the output.
In practice, `top_p` values between 0.7 and 0.9 are often used to balance between diversity and relevance of the output. For example, a `top_p` of 0.85 means the model will randomly choose the next word from a set of top candidates that have a combined probability of 85%.
As with `top_k`, the optimal `top_p` value can depend on your specific use case and the desired behavior of the model.
Generally its not recommended to use Top_P together with the 'temperature' value as the result is difficult to predict.
Here is a more detailed explanation and comparison of the `top_k`, `top_p`, and `temperature` parameters in the context of OpenAI's GPT-3 API.
When GPT-3 generates text, it does so word by word. For each word it generates, it calculates a probability for every word in its vocabulary, and then selects the next word based on these probabilities. The `top_k`, `top_p`, and `temperature` parameters are all ways to influence this selection process.
1. `top_k`: This parameter limits the number of words that the model considers as the next possible word. If `top_k` is set to 50, for example, the model will only consider the 50 words it thinks are most likely. This can make the output more focused and less random, because it's only choosing from a subset of words. However, it can also make the output less diverse, because it's ignoring a lot of potential words. The `top_k` value can be any non-negative integer, with larger values leading to more randomness and smaller values leading to less randomness. If `top_k` is not set, the model considers all possible words.
2. `top_p`: Also known as nucleus sampling, this parameter is a bit more dynamic. Instead of always considering a fixed number of words like `top_k`, `top_p` considers however many words are needed to reach a certain cumulative probability. For example, if `top_p` is set to 0.9, the model will consider the smallest set of words that have a combined probability of 90%. This set of words can be larger or smaller depending on the specific probabilities for each word. Like `top_k`, `top_p` can make the output more focused and less random, but it can also reduce diversity. The `top_p` value is a float between 0 and 1, with larger values leading to more randomness and smaller values leading to less randomness.
3. `temperature`: This parameter controls the "sharpness" of the probability distribution. If `temperature` is set to a high value (close to 1), the model's word selection will be more random and less deterministic, even if some words have much higher probabilities than others. If `temperature` is set to a low value (close to 0), the model's word selection will be more deterministic and less random, with the model strongly favoring words that have higher probabilities. In other words, a high `temperature` makes the model more "adventurous" in its word choices, while a low `temperature` makes the model more "conservative".
In summary, `top_k` and `top_p` are ways to limit the number of words that the model considers for each step of the generation process, while `temperature` is a way to control the randomness of the model's word selection within those limits. All three parameters can be used together to finely tune the behavior of the model. The optimal values for these parameters can depend on your specific use case and the desired behavior of the model. It's a good idea to experiment with different values to see what works best for your needs.
Syntax
AIC.Set Top_P[|P1]
AIC.STP[|P1]
Parameter Explanation
P1 - (optional) numeric value, between 0 and 1. If omitted or -1, then the parameter is not used therefore the System will use internal default values.
Example
'***********************************
'
'***********************************
Remarks
-
Limitations:
-
See also:
•