Parameters

Weave's LLM Parameters

Weave allows the fine-tuning of LLMs through the parameters below:

stop

• What it does: Set custom stop tokens for the generation.

• Values: Strings

top_k

• What it does: Selecting the next token randomly from a specified number, k, of tokens with the highest probabilities.

• Higher values: Lead to greater variability

• Lower values: Lead to lesser variability

top_p

• What it does: Select the next token randomly from the smallest set of tokens for which the cumulative probability exceeds a specified value, p.

• Higher values: Lead to greater variability.

• Lower values: Reduce diversity and focus on more probable tokens

truncate (Non-OpenAI's LLMs only)

• What it does: Truncate inputs tokens to the given size.

• Higher values: Cuts down less texts

• Lower values: Cuts down more texts

do_sample (Non-OpenAI's LLMs only)

• What it does: Activate logits sampling and modify the likelihood of specified tokens appearing in the completion.

• Values: TRUE or FALSE

typical_p (Non-OpenAI's LLMs only)

• What it does: Ensures the responses are more typical or expected.

• Higher values: Less unusual output

• Lower values: More unusual output

watermark (Non-OpenAI's LLMs only)

• What it does: Watermark to determine if the text/image is generated by an AI. Useful to prevent overtraining of LLM’s with generated data.

• Values: TRUE or FALSE

max_tokens

• What it does: The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.

• Higher values: More characters in the output

• Lower values: Less characters in the output

temperature

• What it does: Control the randomness of output.

• Higher values: More random output

• Lower values: More focused output

return_full_text (Non-OpenAI's LLMs only)

• What it does: Whether to prepend the prompt to the generated text.

• Values: TRUE or FALSE

frequency_penalty

• What it does: Control sampling repetitive sequences of tokens

• Higher values: Decreasing the model's likelihood to repeat the same line verbatim

• Lower values: Increasing the model's likelihood to repeat the same line verbatim

presence_penalty (OpenAI's LLMs only)

• What it does: Control sampling repetitive sequences of tokens

• Higher values: Increasing the model's likelihood to talk about new topics

• Lower values: Decreasing the model's likelihood to talk about new topics