promptfoo-evaluation — agentic threat model

Utilizes LLMs for generating configurations, writing Python assertions, and acting as an 'llm-rubric' judge. Vulnerable to prompt injection that could manipulate evaluation results or influence generated Python code.

Handles test cases, few-shot examples, and evaluation datasets. Vulnerable to data poisoning where malicious test cases or assertions are injected to bypass security evaluations.

Orchestrates the generation of promptfooconfig.yaml and Python scripts. Insecure tool integration is a major threat here, as the framework must safely handle and execute generated code.

Runs Promptfoo and Python evaluation scripts directly on the host. This presents a severe threat of host compromise, privilege escalation, and arbitrary code execution if the generated scripts are not strictly sandboxed.

Acts as an evaluation tool itself, but is vulnerable to evaluation gaming or blind spots if the 'llm-rubric' or Python assertions are manipulated to falsely report successful security passes.

Not certain from the listing — there is no mention of built-in authentication, access controls, or policy enforcement mechanisms to restrict who can run host-level evaluation scripts.

Not certain from the listing — the agent appears to operate as a standalone utility for prompt testing, with no explicit multi-agent coordination or ecosystem marketplace interactions described.