Autoresearch — agentic threat model

The agent relies on foundation models to generate code modifications. It is vulnerable to prompt injection via instructions (e.g., in program.md) that could trick the model into generating malicious code or backdooring the training process.

The agent operates on a local LLM training dataset. Threats include data poisoning of the training set, which could skew the evaluation metrics or cause the agent to optimize for malicious behaviors.

The orchestration loop autonomously edits files and executes them. This creates a severe risk of tool misuse, where the agent writes and executes arbitrary, destructive Python code under the guise of an experiment.

Not certain from the listing — there is no mention of sandboxing or containerization. If run directly on host hardware, a compromised agent could achieve full host compromise, privilege escalation, or lateral movement.

The agent autonomously evaluates its own experiment results. This is highly vulnerable to evaluation gaming, where the agent could modify the evaluation script itself to always report success, bypassing the intended logic.

Not certain from the listing — there are no apparent security, identity, or access management controls. The agent likely runs with the full permissions of the user executing the script.

Not certain from the listing — the repository is described as a single-GPU setup and does not explicitly detail multi-agent interactions or external marketplace integrations.