Skip to main content

Get Eval Dataset Execution Result

GET 
/api/v1/pipelines/:pipeline_id/eval-datasets/:eval_dataset_id/execute/result

Get the result of an EvalDatasetExecution. If eval_question_ids is specified, only the results for the specified questions will be returned. If any of the specified questions do not have a result, they will be ignored.

Request​

Path Parameters

    pipeline_id uuidrequired
    eval_dataset_id uuidrequired

Cookie Parameters

    session any

Responses​

Successful Response

Schema

  • Array [

    • eval_question_id uuidrequired

    The ID of the question that was executed.

    pipeline_id uuidrequired

    The ID of the pipeline that the question was executed against.

    source_nodes

    object[]

    required

    The nodes retrieved by the pipeline for the given question.

  • Array [

    • id_ Id (string)

    Unique ID of the node.

    embedding

    object

    Embedding of the node.

    anyOf

  • Array [

    • number

  • ]

    • extra_info object

    A flat dictionary of metadata fields

    excluded_embed_metadata_keys string[]

    Metadata keys that are excluded from text for the embed model.

    excluded_llm_metadata_keys string[]

    Metadata keys that are excluded from text for the LLM.

    relationships

    object

    A mapping of relationships to other node information.

    anyOf
    text Text (string)

    Text content of the node.

    mimetype Mimetype (string)

    Default value: text/plain

    MIME type of the node content.

    start_char_idx

    object

    Start char index of the node.

    anyOf

    integer

    end_char_idx

    object

    End char index of the node.

    anyOf

    integer

    text_template Text Template (string)

    Default value: `{metadata_str}

    {content}`

    Template for how text is formatted, with {content} and {metadata_str} placeholders.

    metadata_template Metadata Template (string)

    Default value: {key}: {value}

    Template for how metadata is formatted, with {key} and {value} placeholders.

    metadata_seperator Metadata Seperator (string)

    Default value:

    Separator between metadata fields when converting to string.

    class_name Class Name (string)

    Default value: TextNode

  • ]

    • answer Answer (string)required

    The answer to the question.

    eval_metrics

    object

    required

    The eval metrics for the question.

    property name*

    MetricResult

    passing

    object

    Whether the metric passed or not.

    anyOf

    boolean

    score

    object

    The score for the metric.

    anyOf

    number

    feedback

    object

    The reasoning for the metric.

    anyOf

    string

    eval_dataset_execution_id uuidrequired

    The ID of the EvalDatasetJobRecord that this result was generated from.

    eval_dataset_execution_params

    object

    required

    Schema for the params for an eval execution.

    llm_model SupportedLLMModelNames (string)

    Possible values: [GPT_3_5_TURBO, GPT_4, GPT_4_TURBO, GPT_4O, GPT_4O_MINI, AZURE_OPENAI]

    Default value: GPT_4O

    The LLM model to use within eval execution.

    qa_prompt_tmpl Qa Prompt Tmpl (string)

    Default value: `Context information is below.

    {context_str}

    Given the context information and not prior knowledge, answer the query. Query: {query_str} Answer: `

    The template to use for the question answering prompt.

    eval_finished_at date-timerequired

    The timestamp when the eval finished.

    class_name Class Name (string)

    Default value: EvalQuestionResult

  • ]

Loading...