sync-provider-contract-docs: Clarify sync provider contract docs #3

Open

std-bot wants to merge 1 commit from task/sync-provider-contract-docs into main

pull from: task/sync-provider-contract-docs

merge into: snorkel:main

snorkel:main

snorkel:task/deleted-file-archaeology-guide

snorkel:task/vue-template-scaffold

Conversation 0 Commits 1 Files changed 6 +234 -47

std-bot commented 2026-06-08 22:05:30 +00:00

First-time contributor

Copy link

Assigned idea

Benchmark authoring specification.

Summary: Update sync.md to remove a stale TODO marker and make the provider module contract explicit for model sync maintainers.

Task intent: Perform an upstream-facing documentation cleanup in this repo: clarify what sync provider modules are responsible for versus what the centralized runner owns, using the existing sync.md model sync documentation as the source of truth.

Benchmark plan: Construct the task package so the solver works in the repo checkout at the fixed commit and is asked to edit only sync.md. The prompt should require removing the literal TODO: delete line and adding or tightening a provider contract section that states provider modules live in packages/core/src/sync/providers/, export an object satisfying SyncProvider<SourceModel>, and only fetch, parse, and translate source models while runner-owned behavior remains in packages/core/src/sync/index.ts.

Environment plan: The environment/repo is the unmodified anomalyco/models.dev checkout at fe7927f2dd5e0a5c576b490af9a69ffc4a7d070b. The task/environment should add only hidden verification assets such as test_outputs.py; no network access, Bun install, or model sync execution is required because the expected upstream change is documentation-only.

Verifier plan: Implement test_outputs.py to read sync.md and fail if TODO: delete remains, if any file other than sync.md is changed according to git diff --name-only, or if the document does not include the required provider contract facts: packages/core/src/sync/providers/, SyncProvider<SourceModel>, fetch/parse/translate language, runner-owned file IO or TOML formatting, and both base_model and base_model_omit.

Oracle plan: Create a reference solution patch that edits sync.md only: delete the stale TODO line near the top and add a concise ## Provider Modules or equivalent subsection explaining the provider contract and the boundary with packages/core/src/sync/index.ts. The oracle can be stored as a golden patch plus the same semantic assertions used by test_outputs.py.

Difficulty plan: Coding agents may over-edit the sync implementation instead of performing the requested documentation change, may remove or rewrite existing command documentation, or may miss subtle contract details such as local base_model / base_model_omit metadata handling and runner-owned deletion/reporting behavior.

Repo patch scope: Expected upstream edits are limited to sync.md only.

Anti-leakage notes: The solver-facing prompt and files must not mention benchmark, harness, Snorkel, Harbor, std-ts, agent, stage, attempt, retry, generated task, or evaluation context.

Approach: Ask for a focused documentation patch: preserve the existing model sync command and runner responsibility sections, remove the stale TODO, and add concrete provider module guidance grounded in the current sync architecture.

Verification: Run python test_outputs.py; it should inspect sync.md and use git diff --name-only to confirm the documentation-only patch and required sync-provider contract wording.

Risk: Low technical risk because the change is documentation-only; the main risk is making the verifier too dependent on exact prose rather than checking the concrete required facts.

Path rules: task package is tasks/; mutable upstream checkout is tasks//task/environment/repo; benchmark package files are under tasks//task; verifier/container environment belongs under tasks//task/environment; regular and UI verifier logic belongs in task/tests/test_outputs.py and test.sh only runs that file through pytest.

Agent context

Task slug: sync-provider-contract-docs
Branch: task/sync-provider-contract-docs

## Assigned idea Benchmark authoring specification. Summary: Update `sync.md` to remove a stale TODO marker and make the provider module contract explicit for model sync maintainers. Task intent: Perform an upstream-facing documentation cleanup in this repo: clarify what sync provider modules are responsible for versus what the centralized runner owns, using the existing `sync.md` model sync documentation as the source of truth. Benchmark plan: Construct the task package so the solver works in the repo checkout at the fixed commit and is asked to edit only `sync.md`. The prompt should require removing the literal `TODO: delete` line and adding or tightening a provider contract section that states provider modules live in `packages/core/src/sync/providers/`, export an object satisfying `SyncProvider<SourceModel>`, and only fetch, parse, and translate source models while runner-owned behavior remains in `packages/core/src/sync/index.ts`. Environment plan: The environment/repo is the unmodified `anomalyco/models.dev` checkout at `fe7927f2dd5e0a5c576b490af9a69ffc4a7d070b`. The task/environment should add only hidden verification assets such as `test_outputs.py`; no network access, Bun install, or model sync execution is required because the expected upstream change is documentation-only. Verifier plan: Implement `test_outputs.py` to read `sync.md` and fail if `TODO: delete` remains, if any file other than `sync.md` is changed according to `git diff --name-only`, or if the document does not include the required provider contract facts: `packages/core/src/sync/providers/`, `SyncProvider<SourceModel>`, fetch/parse/translate language, runner-owned file IO or TOML formatting, and both `base_model` and `base_model_omit`. Oracle plan: Create a reference solution patch that edits `sync.md` only: delete the stale TODO line near the top and add a concise `## Provider Modules` or equivalent subsection explaining the provider contract and the boundary with `packages/core/src/sync/index.ts`. The oracle can be stored as a golden patch plus the same semantic assertions used by `test_outputs.py`. Difficulty plan: Coding agents may over-edit the sync implementation instead of performing the requested documentation change, may remove or rewrite existing command documentation, or may miss subtle contract details such as local `base_model` / `base_model_omit` metadata handling and runner-owned deletion/reporting behavior. Repo patch scope: Expected upstream edits are limited to `sync.md` only. Anti-leakage notes: The solver-facing prompt and files must not mention benchmark, harness, Snorkel, Harbor, std-ts, agent, stage, attempt, retry, generated task, or evaluation context. Approach: Ask for a focused documentation patch: preserve the existing model sync command and runner responsibility sections, remove the stale TODO, and add concrete provider module guidance grounded in the current sync architecture. Verification: Run `python test_outputs.py`; it should inspect `sync.md` and use `git diff --name-only` to confirm the documentation-only patch and required sync-provider contract wording. Risk: Low technical risk because the change is documentation-only; the main risk is making the verifier too dependent on exact prose rather than checking the concrete required facts. Path rules: task package is tasks/<slug>; mutable upstream checkout is tasks/<slug>/task/environment/repo; benchmark package files are under tasks/<slug>/task; verifier/container environment belongs under tasks/<slug>/task/environment; regular and UI verifier logic belongs in task/tests/test_outputs.py and test.sh only runs that file through pytest. ## Agent context Task slug: sync-provider-contract-docs Branch: task/sync-provider-contract-docs <!-- std-ts:idea-pr-body -->

std-bot added 1 commit 2026-06-08 22:05:30 +00:00

std: sync sync-provider-contract-docs 7d5c48e85d

This pull request can be merged automatically.

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin task/sync-provider-contract-docs:task/sync-provider-contract-docs

git switch task/sync-provider-contract-docs

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main

git merge --no-ff task/sync-provider-contract-docs

git switch task/sync-provider-contract-docs

git rebase main

git switch main

git merge --ff-only task/sync-provider-contract-docs

git switch task/sync-provider-contract-docs

git rebase main

git switch main

git merge --no-ff task/sync-provider-contract-docs

git switch main

git merge --squash task/sync-provider-contract-docs

git switch main

git merge --ff-only task/sync-provider-contract-docs

git switch main

git merge task/sync-provider-contract-docs

git push origin main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

snorkel/root-repository-maintenance-s214411!3

No description provided.