Skip to main content

Execute Eval Dataset

POST 
/api/v1/pipelines/:pipeline_id/eval-datasets/:eval_dataset_id/execute

Execute a dataset.

Request​

Path Parameters

    eval_dataset_id uuidrequired
    pipeline_id uuidrequired

Cookie Parameters

    session any

Body

required

    eval_question_ids uuid[]required

    params

    object

    The parameters for the eval execution that will override the ones set in the pipeline.

    llm_model

    object

    The LLM model to use within eval execution.

    anyOf

    string

    Possible values: [GPT_3_5_TURBO, GPT_4, GPT_4_TURBO, GPT_4O, GPT_4O_MINI, AZURE_OPENAI, CLAUDE_3_5_SONNET]

    qa_prompt_tmpl

    object

    The template to use for the question answering prompt.

    anyOf

    string

Responses​

Successful Response

Schema

    job_name Job Name (string)required

    partitions

    object

    required

    The partitions for this execution. Used for determining where to save job output.

    property name*

    object

    anyOf

    string

    parameters

    object

    Additional input parameters for the eval execution.

    anyOf

    Schema for the parameters of an eval dataset job.

    eval_question_ids uuid[]required

    The IDs for the EvalQuestions this execution ran against.

    eval_execution_params

    object

    required

    Schema for the params for an eval execution.

    llm_model SupportedLLMModelNames (string)

    The LLM model to use within eval execution.

    Possible values: [GPT_3_5_TURBO, GPT_4, GPT_4_TURBO, GPT_4O, GPT_4O_MINI, AZURE_OPENAI, CLAUDE_3_5_SONNET]

    Default value: GPT_4O
    qa_prompt_tmpl Qa Prompt Tmpl (string)

    The template to use for the question answering prompt.

    Default value: Context information is below. --------------------- {context_str} --------------------- Given the context information and not prior knowledge, answer the query. Query: {query_str} Answer:

    session_id

    object

    The upstream request ID that created this job. Used for tracking the job across services.

    anyOf

    string

    correlation_id

    object

    The correlation ID for this job. Used for tracking the job across services.

    anyOf

    string

    parent_job_execution_id

    object

    The ID of the parent job execution.

    anyOf

    string

    user_id

    object

    The ID of the user that created this job

    anyOf

    string

    created_at date-time

    Creation datetime

    project_id

    object

    The ID of the project this job belongs to.

    anyOf

    string

    id uuid

    Unique identifier

    status StatusEnum (string)required

    Enum for representing the status of a job

    Possible values: [PENDING, SUCCESS, ERROR, PARTIAL_SUCCESS]

    error_code

    object

    anyOf

    string

    error_message

    object

    anyOf

    string

    attempts

    object

    The number of times this job has been attempted

    anyOf

    integer

    started_at

    object

    anyOf

    string

    ended_at

    object

    anyOf

    string

    updated_at date-time

    Update datetime

    data

    object

    Additional metadata for the job execution.

    anyOf

    object