This article concisely guides the developer settings in Certara's CoAuthor AI Chat. It covers essential parameters like Max Tokens, Temperature, Top K, and Top P, with practical use cases and best practices. Learn how to manage text summarization, table embedding strategies, and system prompts. Discover how to enhance content relevance with RAG queries and optimize retrieval settings, including Num Hits, Heat Ratio, and Re-ranking.

Enables text streaming when prompting. When enabled, text will be returned as soon as the model responds with it.

Enabling reasoning will include step-by-step explanations and justifications in the generated response, providing transparency into the AI's thought process.

Enabling this setting shows citations in the response. Citations are only available when RAG is performed.

This parameter sets the maximum number of tokens (words or characters) that the model can generate in a single response.​

Example: If set to 100, the model will generate up to 100 tokens.

Use Case: Control the length of the output to fit within specific constraints.​

Best Practice: Set a reasonable limit to avoid overly long responses that may not be useful.​

Controls the randomness of the model's output. A lower temperature (e.g., 0.2) makes the output more deterministic, while a higher temperature (e.g., 0.8) makes it more random and creative.​

Example: A temperature of 0.2 produces focused responses, while 0.8 produces more varied responses.

Use Case: Adjust the creativity of the responses.​

Best Practice: Use lower values for factual responses and higher values for creative tasks.​

Limits the model to consider only the top K most probable next tokens​.

Example: If Top K is set to 50, the model considers the 50 most likely next tokens.

Use Case: Focus the model on the most likely next words.​

Best Practice: Use lower values for more focused responses and higher values for more diverse outputs.​

Also known as nucleus sampling, this parameter considers the smallest set of tokens whose cumulative probability exceeds the probability.​

Example: A Top P of 0.9 means the model considers tokens that together have a 90% probability of being the next token.

Use Case: Balance between diversity and focus in the output.​

Best Practice: Set around 0.9 for a good balance between accuracy and creativity.​

Defines the size of chunks when breaking up text for summarization.​

Example: If set to 200, each chunk will contain 200 tokens.

Use Case: Manage large documents by summarizing them in chunks.​

Best Practice: Adjust based on the document size to ensure each chunk is manageable.

Sets the amount of overlap between chunks when summarizing​.

Example: If set to 50, each chunk will overlap by 50 tokens with the previous chunk.

Use Case: Maintain context across chunks.​

Best Practice: Use overlap to ensure continuity in the summary.​

When enabled, tells the model to skip using Retrieval-Augmented Generation (RAG)​.

​

Example: Enable for short documents where the entire text fits into the context window.

Use Case: Useful for short documents where the entire text fits into the context window.​

Best Practice: Enable short texts to simplify processing.​

Determines how tables are embedded. Options include row-by-row embedding and whole table embedding​.

Example: Choose row by row for detailed analysis and the whole table for overall context.

Use Case: Choose based on whether individual rows or the entire table context is more important.​

Best Practice: Use row by row for detailed analysis and the whole table for overall context.​

A predefined prompt that sets the context or behavior for the model​.

Example: "You are a helpful assistant."

Use Case: Guide the model's responses to align with specific requirements.​

Best Practice: Clearly define the desired behavior or context in the system prompt.​

Provides related text to guide the model's responses without complicating the main prompt​.

Example: Adding a relevant paragraph to improve context.

Use Case: Improve the relevance of the generated content.​

Best Practice: Use related text to enhance the context for better responses.

Configure how the model retrieves relevant documents or data​.

Example: Set index name, lookup filter, and enable re-ranking.

Use Case: Optimize document retrieval for better context.​

Best Practice: Fine-tune settings like index name, lookup filter, and re-ranking.​

Sets the number of documents or data points to retrieve during a search​.

Example: If set to 10, the model retrieves 10 documents.

Use Case: Control the breadth of search results.​

Best Practice: Adjust based on the required depth of information.​

Determines how much additional context is included when retrieving sections of documents​.

Example: Expand sections to capture all relevant information.

Use Case: Ensure comprehensive context in retrieved sections.​

Best Practice: Expand sections to capture all relevant information.​

Adjusts the weighting of different parts of the retrieved documents​.

Example: Set higher for critical sections to improve relevance.

Use Case: Emphasize more relevant sections.​

Best Practice: Set higher for critical sections to improve relevance.​

Controls the proportion of the search space to consider when retrieving documents.​

Example: A higher search ratio means a broader search.

Use Case: Broaden or narrow the search scope.​

Best Practice: Use higher ratios for comprehensive searches and lower for focused searches.

Sets the minimum relevance score for retrieved documents to be considered​.

Example: Filter out documents with a relevance score below 0.5.

Use Case: Filter out less relevant documents.​

Best Practice: Set a threshold to ensure high-quality results.​

​

Enables re-ranking of the retrieved documents based on their relevance to the query​.

Example: Enable re-ranking to prioritize the most relevant documents.

Use Case: Improve the quality of search results.​

Best Practice: Enable re-ranking to prioritize the most relevant documents.​

​

These settings allow developers to fine-tune the behavior of CoAuthor's AI Chat from Certara, optimizing it for various use cases and ensuring high-quality outputs.​