[Internal] LLM

LLM Chat and Completion.

[Internal] LLM Text Completion Stream

POST llm/text-completion-stream/

Note

Authorized Roles: root, manager, backend, frontend

Text completion stream.

Only implements ld+json response type. When an error occurs after the stream has started, and a status code has been sent already, the error payload will be sent in the warnings field of the chunk.

Request JSON Object

• query (string) – Optional.

• model (string) – Optional. LLM model like openai/gpt-3.5-turbo or meta-llama/Llama-3-70b-chat-hf

• retrieval (object) –

  Optional. dynamically retrieve items be used when rendering prompt templates. Can be used for few-shot learning or/and RAG

  Inner fields

  • amt (int) – [min: 0 max: 64] Maximum amount of items retrieved

  • algorithms (string) – Retrieval algorithm

  • property (string) – name of the property to use for retrieval

  • filters (list-of-object) – Filter on item properties [see Filtering on Item Property]

    Inner fields

    • property_name (string) – Item-property name

    • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

    • value (json-value) – Filter Value

  • reranking (list-of-object) – Reranking [see Reranking on Item Property]

    Inner fields

    • property_name (string) – Item-property name. Used for diversity reranking rules.

    • op (enum) – choices: [diversity] Reranking op

    • weight (float) – Weight to apply to diversity reranking rules; must be positive; defaults to 1; values between 0 and 1.5 are recommended

    • options (object) – Op-specific options

      Inner fields

      • activation_period (int) –

      • max_activation_count (int) –

      • default_malus (float) –

      • filters (list-of-object) – [internal]

        Inner fields

        • property_name (string) – Item-property name

        • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

        • value (json-value) – Filter Value

      • rank_divider (float) – [internal]

      • similarity (object) – [internal]

      • malus (object) – [internal]

      • rank_scores (object) – [internal]

      • randomness (object) – [internal]

  • candidates_preselection (object) – [Advanced Usage] Candidates preselection

    Inner fields

    • method (string) –

    • params (object) –

    • item_selector_config (object) –

  • properties_returned (list-of-string) – List of items properties to return for items retrieved

• system_prompt (string) – Optional. system prompt template in Jinja

• context_vars (json-value) – Optional. Context variables used when rendering prompt templates

• max_tokens (int) – Optional. [min: 0 max: 100000.0] Maximum tokens for response

• temperature (float) – Optional. [min: 0 max: 1] Sampling temperature controlling how deterministic the output will be.

• scenario (string) – Optional. Name of scenario to apply [see About Scenarios]

• skip_default_scenario (bool) – Optional. Specify whether default scenario should by applied or skipped [see Default Scenario]

• user_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] User ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a session_id.

• session_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] Anonymous Session ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a user_id.

EXAMPLE QUERY BODY

  {
      "query": "The capital of France is"
  }

Response JSON Object

• warnings (list-of-string) – Optional. List of warnings

• response (string) –

• items (list-of-object) – Optional. Retrieved items ID and properties

• evaluated_scenarios (object) –

  Optional.

  Inner fields

  • runtime (list-of-object) – List of evaluated scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • default (list-of-object) – List of evaluated scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • reco_type (enum) – choices: [item_to_items, profile_to_items, session_to_items, profile_to_item_properties, profile_to_items_w_ctx_items, session_to_items_w_ctx_items, precomputed_item_to_items, precomputed_profile_to_items, session_to_item_properties, profiles_group_to_items, chat_to_items, item_properties_to_items, generic_input_to_items, llm_text_completion, llm_chat_completion] Type of recommendation

  • generic_runtime (list-of-object) – List of evaluated generic scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • generic_default (list-of-object) – List of evaluated generic scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

• usage (object) –

  Optional.

  Inner fields

  • prompt_tokens (int) – Number of prompt tokens used

  • completion_tokens (int) – Number of completion tokens used

EXAMPLE RESPONSE CHUNK

  {
      "response": " Paris"
  }

[Internal] LLM Chat Completion Stream

POST llm/chat-completion-stream/

Note

Authorized Roles: root, manager, backend, frontend

Chat completion stream.

Only implements ld+json response type. When an error occurs after the stream has started, and a status code has been sent already, the error payload will be sent in the warnings field of the chunk.

Request JSON Object

• query (list-of-object) –

  Optional. Either a string, or a list-of-objects like openai format

  Inner fields

  • type (string) –

  • text (string) –

  • image_url (object) –

    Inner fields

    • url (string) –

• model (string) – Optional. LLM model like openai/gpt-3.5-turbo or meta-llama/Llama-3-70b-chat-hf

• history (list-of-object) –

  Optional. Chat history

  Inner fields

  • role (string) – for instance user, assistant or system

  • content (list-of-object) –

    Inner fields

    • type (string) –

    • text (string) –

    • image_url (object) –

      Inner fields

      • url (string) –

• retrieval (object) –

  Optional. dynamically retrieve items be used when rendering prompt templates. Can be used for few-shot learning or/and RAG

  Inner fields

  • amt (int) – [min: 0 max: 64] Maximum amount of items retrieved

  • algorithms (string) – Retrieval algorithm

  • property (string) – name of the property to use for retrieval

  • filters (list-of-object) – Filter on item properties [see Filtering on Item Property]

    Inner fields

    • property_name (string) – Item-property name

    • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

    • value (json-value) – Filter Value

  • reranking (list-of-object) – Reranking [see Reranking on Item Property]

    Inner fields

    • property_name (string) – Item-property name. Used for diversity reranking rules.

    • op (enum) – choices: [diversity] Reranking op

    • weight (float) – Weight to apply to diversity reranking rules; must be positive; defaults to 1; values between 0 and 1.5 are recommended

    • options (object) – Op-specific options

      Inner fields

      • activation_period (int) –

      • max_activation_count (int) –

      • default_malus (float) –

      • filters (list-of-object) – [internal]

        Inner fields

        • property_name (string) – Item-property name

        • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

        • value (json-value) – Filter Value

      • rank_divider (float) – [internal]

      • similarity (object) – [internal]

      • malus (object) – [internal]

      • rank_scores (object) – [internal]

      • randomness (object) – [internal]

  • history_injection (object) – [WORK-IN-PROGRESS] inject retrieved items as history messages

    Inner fields

    • json_template (string) – [DEPRECATED] Jinja template rendering to JSON

    • python_literal_template (string) – [WORK-IN-PROGRESS] Jinja template rendering to python literal

  • candidates_preselection (object) – [Advanced Usage] Candidates preselection

    Inner fields

    • method (string) –

    • params (object) –

    • item_selector_config (object) –

  • properties_returned (list-of-string) – List of items properties to return for items retrieved

• system_prompt (string) – Optional. system prompt template in Jinja

• context_vars (json-value) – Optional. Context variables used when rendering prompt templates

• max_tokens (int) – Optional. [min: 0 max: 10000] Maximum tokens for response

• temperature (float) – Optional. [min: 0 max: 1] Sampling temperature controlling how deterministic the output will be.

• scenario (string) – Optional. Name of scenario to apply [see About Scenarios]

• skip_default_scenario (bool) – Optional. Specify whether default scenario should by applied or skipped [see Default Scenario]

• user_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] User ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a session_id.

• session_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] Anonymous Session ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a user_id.

EXAMPLE QUERY BODY

  {
      "query": "What is the capital of France?"
  }

Response JSON Object

• warnings (list-of-string) – Optional. List of warnings

• response (string) –

• items (list-of-object) – Optional. Retrieved items ID and properties

• evaluated_scenarios (object) –

  Optional.

  Inner fields

  • runtime (list-of-object) – List of evaluated scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • default (list-of-object) – List of evaluated scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • reco_type (enum) – choices: [item_to_items, profile_to_items, session_to_items, profile_to_item_properties, profile_to_items_w_ctx_items, session_to_items_w_ctx_items, precomputed_item_to_items, precomputed_profile_to_items, session_to_item_properties, profiles_group_to_items, chat_to_items, item_properties_to_items, generic_input_to_items, llm_text_completion, llm_chat_completion] Type of recommendation

  • generic_runtime (list-of-object) – List of evaluated generic scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • generic_default (list-of-object) – List of evaluated generic scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

• usage (object) –

  Optional.

  Inner fields

  • prompt_tokens (int) – Number of prompt tokens used

  • completion_tokens (int) – Number of completion tokens used

EXAMPLE RESPONSE PART

  {
      "response": "The capital of France is Paris."
  }

[Internal] LLM Text Completion

POST llm/text-completion/

Note

Authorized Roles: root, manager, backend, frontend

Text completion

Request JSON Object

• query (string) – Optional.

• model (string) – Optional. LLM model like openai/gpt-3.5-turbo or meta-llama/Llama-3-70b-chat-hf

• retrieval (object) –

  Optional. dynamically retrieve items be used when rendering prompt templates. Can be used for few-shot learning or/and RAG

  Inner fields

  • amt (int) – [min: 0 max: 64] Maximum amount of items retrieved

  • algorithms (string) – Retrieval algorithm

  • property (string) – name of the property to use for retrieval

  • filters (list-of-object) – Filter on item properties [see Filtering on Item Property]

    Inner fields

    • property_name (string) – Item-property name

    • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

    • value (json-value) – Filter Value

  • reranking (list-of-object) – Reranking [see Reranking on Item Property]

    Inner fields

    • property_name (string) – Item-property name. Used for diversity reranking rules.

    • op (enum) – choices: [diversity] Reranking op

    • weight (float) – Weight to apply to diversity reranking rules; must be positive; defaults to 1; values between 0 and 1.5 are recommended

    • options (object) – Op-specific options

      Inner fields

      • activation_period (int) –

      • max_activation_count (int) –

      • default_malus (float) –

      • filters (list-of-object) – [internal]

        Inner fields

        • property_name (string) – Item-property name

        • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

        • value (json-value) – Filter Value

      • rank_divider (float) – [internal]

      • similarity (object) – [internal]

      • malus (object) – [internal]

      • rank_scores (object) – [internal]

      • randomness (object) – [internal]

  • candidates_preselection (object) – [Advanced Usage] Candidates preselection

    Inner fields

    • method (string) –

    • params (object) –

    • item_selector_config (object) –

  • properties_returned (list-of-string) – List of items properties to return for items retrieved

• system_prompt (string) – Optional. system prompt template in Jinja

• context_vars (json-value) – Optional. Context variables used when rendering prompt templates

• max_tokens (int) – Optional. [min: 0 max: 100000.0] Maximum tokens for response

• temperature (float) – Optional. [min: 0 max: 1] Sampling temperature controlling how deterministic the output will be.

• scenario (string) – Optional. Name of scenario to apply [see About Scenarios]

• skip_default_scenario (bool) – Optional. Specify whether default scenario should by applied or skipped [see Default Scenario]

• user_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] User ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a session_id.

• session_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] Anonymous Session ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a user_id.

EXAMPLE QUERY BODY

  {
      "query": "The capital of France is"
  }

Response JSON Object

• warnings (list-of-string) – Optional. List of warnings

• response (string) –

• items (list-of-object) – Optional. Retrieved items ID and properties

• evaluated_scenarios (object) –

  Optional.

  Inner fields

  • runtime (list-of-object) – List of evaluated scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • default (list-of-object) – List of evaluated scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • reco_type (enum) – choices: [item_to_items, profile_to_items, session_to_items, profile_to_item_properties, profile_to_items_w_ctx_items, session_to_items_w_ctx_items, precomputed_item_to_items, precomputed_profile_to_items, session_to_item_properties, profiles_group_to_items, chat_to_items, item_properties_to_items, generic_input_to_items, llm_text_completion, llm_chat_completion] Type of recommendation

  • generic_runtime (list-of-object) – List of evaluated generic scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • generic_default (list-of-object) – List of evaluated generic scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

• usage (object) –

  Optional.

  Inner fields

  • prompt_tokens (int) – Number of prompt tokens used

  • completion_tokens (int) – Number of completion tokens used

EXAMPLE RESPONSE

  {
      "response": " Paris",
      "usage": {
          "prompt_tokens": 5,
          "completion_tokens": 1
      }
  }

[Internal] LLM Chat Completion

POST llm/chat-completion/

Note

Authorized Roles: root, manager, backend, frontend

Chat completion

Request JSON Object

• query (list-of-object) –

  Optional. Either a string, or a list-of-objects like openai format

  Inner fields

  • type (string) –

  • text (string) –

  • image_url (object) –

    Inner fields

    • url (string) –

• model (string) – Optional. LLM model like openai/gpt-3.5-turbo or meta-llama/Llama-3-70b-chat-hf

• history (list-of-object) –

  Optional. Chat history

  Inner fields

  • role (string) – for instance user, assistant or system

  • content (list-of-object) –

    Inner fields

    • type (string) –

    • text (string) –

    • image_url (object) –

      Inner fields

      • url (string) –

• retrieval (object) –

  Optional. dynamically retrieve items be used when rendering prompt templates. Can be used for few-shot learning or/and RAG

  Inner fields

  • amt (int) – [min: 0 max: 64] Maximum amount of items retrieved

  • algorithms (string) – Retrieval algorithm

  • property (string) – name of the property to use for retrieval

  • filters (list-of-object) – Filter on item properties [see Filtering on Item Property]

    Inner fields

    • property_name (string) – Item-property name

    • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

    • value (json-value) – Filter Value

  • reranking (list-of-object) – Reranking [see Reranking on Item Property]

    Inner fields

    • property_name (string) – Item-property name. Used for diversity reranking rules.

    • op (enum) – choices: [diversity] Reranking op

    • weight (float) – Weight to apply to diversity reranking rules; must be positive; defaults to 1; values between 0 and 1.5 are recommended

    • options (object) – Op-specific options

      Inner fields

      • activation_period (int) –

      • max_activation_count (int) –

      • default_malus (float) –

      • filters (list-of-object) – [internal]

        Inner fields

        • property_name (string) – Item-property name

        • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

        • value (json-value) – Filter Value

      • rank_divider (float) – [internal]

      • similarity (object) – [internal]

      • malus (object) – [internal]

      • rank_scores (object) – [internal]

      • randomness (object) – [internal]

  • history_injection (object) – [WORK-IN-PROGRESS] inject retrieved items as history messages

    Inner fields

    • json_template (string) – [DEPRECATED] Jinja template rendering to JSON

    • python_literal_template (string) – [WORK-IN-PROGRESS] Jinja template rendering to python literal

  • candidates_preselection (object) – [Advanced Usage] Candidates preselection

    Inner fields

    • method (string) –

    • params (object) –

    • item_selector_config (object) –

  • properties_returned (list-of-string) – List of items properties to return for items retrieved

• system_prompt (string) – Optional. system prompt template in Jinja

• context_vars (json-value) – Optional. Context variables used when rendering prompt templates

• max_tokens (int) – Optional. [min: 0 max: 10000] Maximum tokens for response

• temperature (float) – Optional. [min: 0 max: 1] Sampling temperature controlling how deterministic the output will be.

• scenario (string) – Optional. Name of scenario to apply [see About Scenarios]

• skip_default_scenario (bool) – Optional. Specify whether default scenario should by applied or skipped [see Default Scenario]

• user_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] User ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a session_id.

• session_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] Anonymous Session ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a user_id.

EXAMPLE QUERY BODY

  {
      "query": "What is the capital of France?"
  }

Response JSON Object

• warnings (list-of-string) – Optional. List of warnings

• response (string) –

• items (list-of-object) – Optional. Retrieved items ID and properties

• evaluated_scenarios (object) –

  Optional.

  Inner fields

  • runtime (list-of-object) – List of evaluated scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • default (list-of-object) – List of evaluated scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • reco_type (enum) – choices: [item_to_items, profile_to_items, session_to_items, profile_to_item_properties, profile_to_items_w_ctx_items, session_to_items_w_ctx_items, precomputed_item_to_items, precomputed_profile_to_items, session_to_item_properties, profiles_group_to_items, chat_to_items, item_properties_to_items, generic_input_to_items, llm_text_completion, llm_chat_completion] Type of recommendation

  • generic_runtime (list-of-object) – List of evaluated generic scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • generic_default (list-of-object) – List of evaluated generic scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

• usage (object) –

  Optional.

  Inner fields

  • prompt_tokens (int) – Number of prompt tokens used

  • completion_tokens (int) – Number of completion tokens used

EXAMPLE RESPONSE

  {
      "response": "The capital of France is Paris.",
      "usage": {
          "prompt_tokens": 5,
          "completion_tokens": 6
      }
  }

[Internal] Get LLM Text Completion request payload

POST llm/text-completion/request-payload/

Note

Authorized Roles: root, manager, backend, frontend

Get text completion request payload

Request JSON Object

• query (string) – Optional.

• model (string) – Optional. LLM model like openai/gpt-3.5-turbo or meta-llama/Llama-3-70b-chat-hf

• retrieval (object) –

  Optional. dynamically retrieve items be used when rendering prompt templates. Can be used for few-shot learning or/and RAG

  Inner fields

  • amt (int) – [min: 0 max: 64] Maximum amount of items retrieved

  • algorithms (string) – Retrieval algorithm

  • property (string) – name of the property to use for retrieval

  • filters (list-of-object) – Filter on item properties [see Filtering on Item Property]

    Inner fields

    • property_name (string) – Item-property name

    • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

    • value (json-value) – Filter Value

  • reranking (list-of-object) – Reranking [see Reranking on Item Property]

    Inner fields

    • property_name (string) – Item-property name. Used for diversity reranking rules.

    • op (enum) – choices: [diversity] Reranking op

    • weight (float) – Weight to apply to diversity reranking rules; must be positive; defaults to 1; values between 0 and 1.5 are recommended

    • options (object) – Op-specific options

      Inner fields

      • activation_period (int) –

      • max_activation_count (int) –

      • default_malus (float) –

      • filters (list-of-object) – [internal]

        Inner fields

        • property_name (string) – Item-property name

        • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

        • value (json-value) – Filter Value

      • rank_divider (float) – [internal]

      • similarity (object) – [internal]

      • malus (object) – [internal]

      • rank_scores (object) – [internal]

      • randomness (object) – [internal]

  • candidates_preselection (object) – [Advanced Usage] Candidates preselection

    Inner fields

    • method (string) –

    • params (object) –

    • item_selector_config (object) –

  • properties_returned (list-of-string) – List of items properties to return for items retrieved

• system_prompt (string) – Optional. system prompt template in Jinja

• context_vars (json-value) – Optional. Context variables used when rendering prompt templates

• max_tokens (int) – Optional. [min: 0 max: 100000.0] Maximum tokens for response

• temperature (float) – Optional. [min: 0 max: 1] Sampling temperature controlling how deterministic the output will be.

• scenario (string) – Optional. Name of scenario to apply [see About Scenarios]

• skip_default_scenario (bool) – Optional. Specify whether default scenario should by applied or skipped [see Default Scenario]

• user_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] User ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a session_id.

• session_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] Anonymous Session ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a user_id.

EXAMPLE QUERY BODY

  {
      "query": "The capital of France is"
  }

Response JSON Object

• warnings (list-of-string) – Optional. List of warnings

• llm_request_payload (object) – Runtime request payload for llm apis

• items (list-of-object) – Optional. Retrieved items ID and properties

• context_vars (json-value) – Optional. Context variables used when rendering prompt templates

• evaluated_scenarios (object) –

  Optional.

  Inner fields

  • runtime (list-of-object) – List of evaluated scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • default (list-of-object) – List of evaluated scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • reco_type (enum) – choices: [item_to_items, profile_to_items, session_to_items, profile_to_item_properties, profile_to_items_w_ctx_items, session_to_items_w_ctx_items, precomputed_item_to_items, precomputed_profile_to_items, session_to_item_properties, profiles_group_to_items, chat_to_items, item_properties_to_items, generic_input_to_items, llm_text_completion, llm_chat_completion] Type of recommendation

  • generic_runtime (list-of-object) – List of evaluated generic scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • generic_default (list-of-object) – List of evaluated generic scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

EXAMPLE RESPONSE

  {
      "llm_request_params": {
              "model": "microsoft/phi-2",
              "prompt": "You are a helpful assistant [...] The capital of France is",
              "max_tokens": 128,
              "temperature": 0.
      },
      "context_vars": {
          "date": "Thursday, May 30st"
      },
      "items": [{"item_id": 1, "my_prop": "val1"}, {"item_id": 2, "my_prop": "val2"}]
  }

[Internal] Get LLM Chat Completion request payload

POST llm/chat-completion/request-payload/

Note

Authorized Roles: root, manager, backend, frontend

Get chat completion request payload

Request JSON Object

• query (list-of-object) –

  Optional. Either a string, or a list-of-objects like openai format

  Inner fields

  • type (string) –

  • text (string) –

  • image_url (object) –

    Inner fields

    • url (string) –

• model (string) – Optional. LLM model like openai/gpt-3.5-turbo or meta-llama/Llama-3-70b-chat-hf

• history (list-of-object) –

  Optional. Chat history

  Inner fields

  • role (string) – for instance user, assistant or system

  • content (list-of-object) –

    Inner fields

    • type (string) –

    • text (string) –

    • image_url (object) –

      Inner fields

      • url (string) –

• retrieval (object) –

  Optional. dynamically retrieve items be used when rendering prompt templates. Can be used for few-shot learning or/and RAG

  Inner fields

  • amt (int) – [min: 0 max: 64] Maximum amount of items retrieved

  • algorithms (string) – Retrieval algorithm

  • property (string) – name of the property to use for retrieval

  • filters (list-of-object) – Filter on item properties [see Filtering on Item Property]

    Inner fields

    • property_name (string) – Item-property name

    • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

    • value (json-value) – Filter Value

  • reranking (list-of-object) – Reranking [see Reranking on Item Property]

    Inner fields

    • property_name (string) – Item-property name. Used for diversity reranking rules.

    • op (enum) – choices: [diversity] Reranking op

    • weight (float) – Weight to apply to diversity reranking rules; must be positive; defaults to 1; values between 0 and 1.5 are recommended

    • options (object) – Op-specific options

      Inner fields

      • activation_period (int) –

      • max_activation_count (int) –

      • default_malus (float) –

      • filters (list-of-object) – [internal]

        Inner fields

        • property_name (string) – Item-property name

        • op (enum) – choices (case insensitive): [eq, lt, lte, gt, gte, in, empty, notempty, neq, notin, ftsearch] Filter Operator

        • value (json-value) – Filter Value

      • rank_divider (float) – [internal]

      • similarity (object) – [internal]

      • malus (object) – [internal]

      • rank_scores (object) – [internal]

      • randomness (object) – [internal]

  • history_injection (object) – [WORK-IN-PROGRESS] inject retrieved items as history messages

    Inner fields

    • json_template (string) – [DEPRECATED] Jinja template rendering to JSON

    • python_literal_template (string) – [WORK-IN-PROGRESS] Jinja template rendering to python literal

  • candidates_preselection (object) – [Advanced Usage] Candidates preselection

    Inner fields

    • method (string) –

    • params (object) –

    • item_selector_config (object) –

  • properties_returned (list-of-string) – List of items properties to return for items retrieved

• system_prompt (string) – Optional. system prompt template in Jinja

• context_vars (json-value) – Optional. Context variables used when rendering prompt templates

• max_tokens (int) – Optional. [min: 0 max: 10000] Maximum tokens for response

• temperature (float) – Optional. [min: 0 max: 1] Sampling temperature controlling how deterministic the output will be.

• scenario (string) – Optional. Name of scenario to apply [see About Scenarios]

• skip_default_scenario (bool) – Optional. Specify whether default scenario should by applied or skipped [see Default Scenario]

• user_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] User ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a session_id.

• session_id (ID) – Optional. [see Flexible Identifiers], [max_length: 64] Anonymous Session ID. Only used in the context of an A/B test scenario to select the group A or B and keep track of the respective group in analytics. NOT used to personalize recommendations. Cannot be used together with a user_id.

EXAMPLE QUERY BODY

  {
      "query": "What is the capital of France?"
  }

Response JSON Object

• warnings (list-of-string) – Optional. List of warnings

• llm_request_payload (object) – Runtime request payload for llm apis

• items (list-of-object) – Optional. Retrieved items ID and properties

• context_vars (json-value) – Optional. Context variables used when rendering prompt templates

• evaluated_scenarios (object) –

  Optional.

  Inner fields

  • runtime (list-of-object) – List of evaluated scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • default (list-of-object) – List of evaluated scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • reco_type (enum) – choices: [item_to_items, profile_to_items, session_to_items, profile_to_item_properties, profile_to_items_w_ctx_items, session_to_items_w_ctx_items, precomputed_item_to_items, precomputed_profile_to_items, session_to_item_properties, profiles_group_to_items, chat_to_items, item_properties_to_items, generic_input_to_items, llm_text_completion, llm_chat_completion] Type of recommendation

  • generic_runtime (list-of-object) – List of evaluated generic scenarios from runtime scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

  • generic_default (list-of-object) – List of evaluated generic scenarios from default scenario [see About Scenarios]

    Inner fields

    • scenario_type (enum) – choices: [case, ab_test, condition, alias] Type of scenario

    • scenario_name (string) – Name of scenario

    • to (string) – Characterization of next scenario evaluated

EXAMPLE RESPONSE

  {
      "llm_request_params": {
              "model": "microsoft/phi-2",
              "messages": [
                  {"role": "system", "content": "You are a helpful assistant"},
                  {"role": "user", "content": "question1"},
                  {"role": "system", "content": "answer1"},
                  {"role": "user", "content": "question2"},
                  {"role": "system", "content": "answer2"},
                  {"role": "user", "content": "What is the capital of France?"}
              ],
              "max_tokens": 128,
              "temperature": 0.
      },
      "context_vars": {
          "date': 'Thursday, May 30st"
      },
      "items": [
          {"item_id": 1, "question": "question1", "answer": "answer1"},
          {"item_id": 2, "question": "question2", "answer": "answer2"}
      ]
  }

[Internal] List all LLM models available for completion endpoints

GET llm/models/

Note

Authorized Roles: root, manager, backend, frontend

List all LLM models available for use in completion endpoints

Response JSON Object

• warnings (list-of-string) – Optional. List of warnings

• models (list-of-object) –

  Inner fields

  • name (string) – Model name

  • type (string) – Model type

  • metadata (object) –

EXAMPLE RESPONSE

  {
      "models": [
          {
              "name": "openai/gpt4",
              "type": "chat_completion",
              "metadata": {}
          },
          {
              "name": "mistralai/Mistral-7B-v0.1",
              "type": "text_completion",
              "metadata": {}
          }
      ]
  }