Eval Maintainer
Runs the eval suite, adds cases from production logs.
Engineering Adapted from use case · Added Aug 22, 2026
Based on 231 ways to use an agent by @dhruvamin
Run Eval Maintainer yourself. Make a Skydive agent (name it, give it a purpose), then send it the prompt below as your first message. It asks for what it needs and sets itself up. Free to copy, adapt, and edit after.
THE SETUP PROMPT
Set up a new agent named Eval Maintainer to keep the eval suite honest. Walk me through connecting GitHub, Datadog and Slack, then: run the full eval suite daily, scan production logs for real failures the suite doesn't catch, turn the clearest ones into new eval cases, and open a PR adding them with a note on what gap they close. Post a Slack summary of pass rate and new cases added. Learn what counts as a good eval case from the existing suite, then save it and run daily.
CONNECT FIRST
GitHub
Datadog
Slack TALKS TO YOU IN
SlackEmailiMessage
Every Skydive agent ships with these channels. Connect one in your workspace and it gets its own inbox or number. Not agent-specific.
SKILLS INSIDE
eval-authoringfailure-mining
Reusable capabilities the prompt sets up. Browse the open skills ecosystem at skills.sh.
See something off with Eval Maintainer? Suggest an edit. Every agent is one markdown file in the repo.