[rig-tasks] Daily rig evaluation — 2026-07-23 — 5/5 passed

[github-actions]

Summary

Task	Description	Typecheck	Key finding
1	Environment variable validator (p.read + p.bash + s.enum)	✅ pass	Clean on first attempt
2	API endpoint extractor with classifier subagent	✅ pass (after fix)	agents must be Record<string, AgentFn>, not an array
3	Package health checker (two-agent chaining)	✅ pass (after fix)	Unused variable error when intermediate agent not exported
4	Git tag annotation summarizer (s.record + s.optional + s.enum)	✅ pass	Clean on first attempt
5	JSON config merger with defineTool	✅ pass (after fix)	defineTool type parameter must be provided explicitly

Problems encountered

Task 2 — agents field requires Record<string, AgentFn>, not an array

What the generated code tried to do: Pass agents: [classifierAgent] (an array) to the agent spec.

What went wrong:

Type 'AgentFn<...>[]' is not assignable to type 'Record<string, AgentFn<any, any>>'.
  Index signature for type 'string' is missing in type 'AgentFn<...>[]'.

Fix: Change agents: [classifierAgent] to agents: { classifier: classifierAgent }.

Observation: The array form is a very natural ergonomic default (the name is already on the agent spec), but the harness requires a plain object. SKILL.md does not clearly document the required shape — it only says agents?: Record<string, AgentFn<any, any>> in the type signature, with no example. A short code example showing agents: { mySubagent } would prevent this error.

---

Task 3 — Unused intermediate agent causes TS6133

What the generated code tried to do: Declare extractorAgent locally (not exported) to document the two-agent chaining pattern.

What went wrong:

TS6133: 'extractorAgent' is declared but its value is never read.

Fix: Export the intermediate agent (export const extractorAgent = agent(...)).

Observation: For chained multi-agent patterns, the natural pattern is to declare both agents in a single file where only the final one is the default export. The TS strict unused-variable check makes this awkward. A convention (documented in SKILL.md) to either export intermediate agents or use them inside the root agent's agents map would clarify the intended pattern.

---

Task 5 — defineTool type parameter not inferred from parameters schema

What the generated code tried to do: Use defineTool("name", { parameters: s.object({ json: s.string }), handler: async ({ json }) => ... }).

What went wrong:

TS2322: Type '({ json }: { json: any; }) => Promise<string>' is not assignable to type 'ToolHandler<unknown>'.
  Types of parameters '__0' and 'args' are incompatible.
    Type 'unknown' is not assignable to type '{ json: any; }'.

Fix: Explicitly specify the type parameter: defineTool<{ json: string }>("name", { ... }).

Observation: The parameters field is Schema | Record<string, unknown> and carries no TypeScript inference into T. This means the developer must duplicate the shape as a generic argument — the schema and the generic are not linked. Ideally, InferSchema<typeof parameters> could be used to derive T automatically, eliminating the manual type annotation.

---

Subagent infrastructure — rig-expander and task-generator agents produced no output

What happened: Both custom agents (rig-expander and task-generator) completed their runs with empty responses across multiple turns. The programs were written manually as a fallback.

Impact: The evaluation pipeline requires these agents to function. Empty agent responses make the workflow brittle.

---

Improvement opportunities

Missing schema helpers (s.*)

• s.infer<typeof schema> (or InferSchema re-exported publicly): Allow deriving the TypeScript type from a schema so defineTool<T> can be written defineTool<s.infer<typeof myParams>> — eliminating duplicate type declarations.
• s.nonEmptyArray(item): Shorthand for arrays that must have at least one element (maps to minItems: 1).
• s.tuple([s.string, s.number]): Fixed-length heterogeneous arrays are common but not expressible.

Missing prompt helpers (p.*)

• p.readAll(glob): Read all files matching a glob pattern and concatenate them — a common pattern that currently requires a p.bash("cat ...") workaround.
• p.readJson(path): Read a JSON file and pretty-print it inline, avoiding the JSON.parse / JSON.stringify round-trip when the goal is just inspection.

API ergonomics

• agents field should accept arrays: agents: [classifierAgent] (where each agent has a name field) is more ergonomic than agents: { classifier: classifierAgent }. The harness could derive the key from agent.spec.name. This is the Import pelikhan/rig into githubnext/rig and harden JSON fallback parsing #1 ergonomic friction point encountered.
• defineTool should infer handler type from parameters: When parameters is a Schema, the harness knows the shape — T should be inferred automatically so developers don't need to repeat <{ json: string }>.
• Multi-agent chaining pattern documentation: SKILL.md should show how to write a file with both an intermediate agent and a root agent, and clarify whether intermediate agents should be exported or included in agents.

Error message quality

• The agents array-vs-record error is clear but not actionable without knowing the correct form. A more helpful message could be: "agents must be a plain object mapping name → AgentFn, e.g. { myAgent }. Received an array.".

Documentation gaps

• agents spec field: SKILL.md shows the type signature but no code example. Add a snippet showing agents: { classifier: classifierAgent } alongside a note that the key is used as the subagent name in prompts.
• Two-agent chaining: No sample documents how to chain two agents where the second takes the first's output. Add a minimal example covering: declare extractorAgent, declare assessorAgent (with input = extractor's output schema), export both, chain externally.
• defineTool generic parameter: SKILL.md example shows defineTool without noting that <T> must be provided explicitly when the handler destructures parameters.

Tasks run today

• (reused) Task 1: Environment variable validator that reads .env.example via p.read, checks actual env via p.bash, and outputs s.array of env var statuses with s.enum ok/missing/extra
• (reused) Task 2: API endpoint extractor that scans route files via p.bash, outputs s.array(s.object) of endpoints, with a classifier subagent using s.enum for purpose
• (reused) Task 3: Package health checker with two-agent chaining: extractor reads package.json via p.read, assessor produces s.enum health status
• (new) Task 4: Git tag annotation summarizer using p.bash git for-each-ref, outputs s.record keyed by tag name with s.object containing s.optional message, date, and s.enum stability (stable/prerelease/unknown)
• (new) Task 5: JSON config merger reading multiple files via p.read and p.readOptional, uses defineTool for JSON validation, outputs s.record for merged keys, s.array for conflicts, and fileCount

Generated by Daily Rig Task Generator · sonnet46 65.1 AIC · ⌖ 4.76 AIC · ⊞ 5.5K · ◷