How many tokens the model is allowed to generate before being stopped.
Note that this only controls how many tokens can be generated (before the response is cut off), not how many will be generated.
Setting this to a high value will not make the replies more verbose, and conversely, setting it to a low value will not make the replies more concise.
The maximum value is 128K tokens (128 * 1024 = 131072)
temperaturenumber 路 max: 2 路 default: 1
Temperature controls how much the model is allowed to deviate from standard behavior.
Lower values will make the model more conservative in its responses, and values like 0 will make the model deterministic (i.e. it will always generate the same output for the same input).
Higher values, like 0.8 will make the model more creative in its responses, meaning it will take more risks and generate more unexpected outputs.
Values above 1 are not recommended, as they can lead to nonsensical outputs, but we allow them for experimentation.
Frequency penalty controls how much the model is allowed to repeat itself.
Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the likelihood of the model repeating the same lines over and over.
Presence penalty controls the model's likelihood of talking about new topics.
Positive values penalize tokens based on whether they have already appeared in the text so far, making the model more likely to introduce new topics.
top_pnumber 路 max: 1 路 default: 1
An alternative to sampling with temperature, a top_p value of 0.1 means the model will be forced to choose from the tokens that make up the top 10% of the probability distribution, making the output more conservative and less creative.
Low values of top_p have a similar effect to low values of temperature (i.e. they make the model more conservative).
While we allow you to set both temperature and top_p at the same time, we recommend using only one of them at a time, as they can interfere with each other - this is very likely not what you want to do.
top_knumber
seedinteger 路 min: -1 路 max: 9007199254740991
Seed is an arbitrary integer number that allows you to make the model's output deterministic.
Using the same seed will make the model generate the same output for the same input - though note that this is not always guaranteed, as different versions of the model/hardware/libraries may generate slightly different outputs.
If you do not provide a seed, we will generate one for you, making the output non-deterministic.
Note that lower temperature values will tend to make the output more similar, regardless of the seed.
The model will not call any tool, and will instead reply with a message.
The model may elect to call a function, or reply with a message. Beware that this is an experimental feature: since the model is free to output any text whatsoever, it may attempt to call a function using an invalid syntax.
The model must call one (or multiple) of the tools provided