feat: FTW's field-proven drivers become the source#27
Conversation
This repository held 62 drivers of unknown provenance and treated them as the
truth. FTW held 37 that have run on customer sites for months. Every serious
defect found while working here was in the catalog, never in FTW's drivers: a
sign bug flipping every float in 12 of them, 35 calling host functions that do
not exist, 60 of 62 manifest hashes lying, a Sungrow control path that charges
when told to discharge. The wrong set was being repaired.
So the drivers move rather than converge. All 37 are promoted verbatim: 19
replace a catalog driver of the same name, 18 are new. Byte-identical to
baselines/ftw on purpose -- that is what makes provenance checkable, and what
makes any later edit visible.
Manifests are derived from what each driver already declares in its DRIVER
table rather than retyped, so the two cannot drift.
Three rules turned out to be invented rather than measured, and each was
rewritten to describe what is actually there:
DER vocabulary knew only pv, battery, meter, v2x_charger. FTW also emits
ev, heatpump and vehicle, so eight shipped drivers had no
honest way to describe themselves. myuplink emits only
metrics and now says so with an empty list instead of
claiming telemetry it never sends.
host-API profile called modbus_write and millis "spellings this catalog grew
on its own". They are FTW's real API names. It also omitted
the entire transport layer -- ws_open, tcp_open, mqtt_pub,
persist_secret. The list now comes from FTW's Go binding.
canonical debt measured distance from an ideal that never existed. Deleted.
In its place, tools/host_api_check.py asks the only question
that predicts a crash: does a driver call a function no host
provides? That is what 35 drivers were doing while passing
every test here.
Two identifiers, two lineages, both deliberate. A promoted driver's DRIVER
table states FTW's id and version; the file name and manifest state this
repository's. Forcing them equal would mean editing files we keep unmodified
on purpose, so the checks compare instead of demand.
The suite skips catalog conventions for a driver that is byte-identical to its
baseline, since FTW tests those in Go. Edit one here and every check applies to
it again -- identity is by content, not by a list that would rot. Tests pinning
the behaviour of replaced catalog drivers are removed; they tested code that no
longer exists.
make check: 1918 passed, 80 drivers, sandbox and host-API clean.
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
…ndoned blueprint/BLUEPRINT.lua is a complete working driver for an imaginary inverter. It exists so every rule in this repository has one place to be demonstrated rather than described, and it is written to be read by people and by agents alike: the contract is stated in full, each rule sits beside the code that follows it, and nothing is left implicit. It is verified like a shipped driver rather than trusted. tests/test_blueprint.py compiles it, runs the sandbox and host-API checks on it, and polls it against the harness -- including the test that matters most, that failed reads per poll settle to zero. A blueprint that would take a site offline teaches the mistake it demonstrates. It lives outside drivers/lua so it is checked without being published. docs/WRITING-A-DRIVER.md gives the reasoning: why one failed read can take a whole site offline, why a fabricated zero is worse than a missing field, why splitting a 32-bit value the obvious way flips the sign on a 32-bit build, and why arithmetic never belongs in the host API. Each rule is tied to the incident that produced it. spec/host-api.md still told readers that Blixt's names were canonical and that tools/canonical_debt.py counted what was left to convert. Neither is true: the tool is gone and the two spellings are two real host APIs, not a right and a wrong one. The README claimed one canonical driver per device as the goal; what we actually want is one driver per device and no opinion about which dialect it speaks. Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
… source The recipe declared read_only: true and copied drivers/lua/goodwe.lua straight through. That was true of the catalog's goodwe. It is not true of FTW's, which has a control path and declares nothing of the kind, so the build correctly refused rather than shipping a control-capable driver under a package that promised it could not write. Rebuilding it needs a write-inert target derived on purpose, the way packages/v1/sungrow/targets/ftw-observe.lua was. That is a decision about hardware, not a build fix, so it waits for someone who wants goodwe published and understands its control path. Also collapses changelog entries for tools added and removed inside the same unreleased block. Nothing there shipped, so it is churn rather than history. Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
2ce8303 to
40b9f39
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 037efdaf51
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| host.emit("pv", { | ||
| W = -pv_w, | ||
| mppt1_v = mppt1[1] * 0.1, | ||
| mppt1_a = mppt1[2] * 0.1, | ||
| mppt2_v = mppt2[1] * 0.1, | ||
| mppt2_a = mppt2[2] * 0.1, | ||
| total_generation_Wh = host.decode_u32_be(pv_energy[1], pv_energy[2]) * 100, | ||
| w = -pv_w, -- negative = generation (site convention) |
There was a problem hiding this comment.
When the PV power read at line 51 fails, pv_w remains zero and this unconditional emit publishes it as a fresh zero-power sample; the battery and meter sections repeat the same fail-open pattern. A disconnected or partially failing inverter will therefore appear idle rather than stale, potentially corrupting site calculations and control decisions. Abort the poll or omit each stream unless its required registers were read successfully.
AGENTS.md reference: AGENTS.md:L25-L26
Useful? React with 👍 / 👎.
| local resp, err = host.http_post( | ||
| BASE_URL .. "/oauth/token", body, | ||
| { ["Content-Type"] = "application/x-www-form-urlencoded" }) |
There was a problem hiding this comment.
In the signed FTW channel, _ftw_read_only_artifact() replaces host.http_post with a function that throws before this source loads. With a configured refresh token, driver_init() reaches this unprotected call through try_setup(), raises immediately, and MyUplink can never authenticate or emit its read-only telemetry. Read-only wrapping must distinguish authentication/query POSTs from control POSTs rather than denying this required transport operation.
AGENTS.md reference: AGENTS.md:L15-L16
Useful? React with 👍 / 👎.
| version: "1.0.0" | ||
| tier: core | ||
| author: "Sourceful Labs AB" | ||
| protocol: modbus |
There was a problem hiding this comment.
The Lua driver declares PROTOCOL = "tcp" and polls exclusively through host.tcp_open, host.tcp_is_open, and host.tcp_recv, but this manifest labels it Modbus. The FTW channel derives signed permissions from this field, so it publishes only modbus.read; a host enforcing those permissions cannot open the socket and the driver never receives a telegram. Add TCP to the supported protocol/permission maps and declare it here.
AGENTS.md reference: AGENTS.md:L27-L29
Useful? React with 👍 / 👎.
| version: "1.0.0" | ||
| tier: core | ||
| author: "Sourceful Labs AB" | ||
| protocol: http |
There was a problem hiding this comment.
Tibber declares PROTOCOL = "websocket" and requires ws_open, ws_send, and ws_messages for every telemetry poll, yet this manifest is recorded as HTTP. The channel therefore signs only the http.get permission and omits WebSocket access, so an enforcing host cannot establish the GraphQL subscription even when home_id is preconfigured. Represent WebSocket as a supported protocol and include its required permissions.
AGENTS.md reference: AGENTS.md:L27-L29
Useful? React with 👍 / 👎.
| port = 502, | ||
| unit_id = 1, | ||
| }, | ||
| } |
There was a problem hiding this comment.
The existing packages/v1/goodwe/package-source.json is read-only and targets ftw-core, whose package validator requires the Lua DRIVER table to explicitly declare read_only = true. The replacement table closes here without that field, so make package-driver ID=goodwe TARGET=ftw-core now fails with read-only Lua source must declare read_only = true, breaking the existing GoodWe pilot package.
AGENTS.md reference: AGENTS.md:L27-L29
Useful? React with 👍 / 👎.
| # Two lineages, deliberately. The package version counts releases from this | ||
| # repository; a driver promoted from FTW keeps FTW's DRIVER table verbatim, | ||
| # so its declared version counts FTW's. Rewriting that field would mean | ||
| # editing a file we keep byte-identical to its baseline on purpose, which | ||
| # is how provenance stays checkable. A missing version is still an error. | ||
| if not declared_version: | ||
| raise PackageError(f"{target}: Lua metadata must declare a version") |
There was a problem hiding this comment.
Removing the equality check permits signed packages whose embedded DRIVER.version disagrees with the package identity; for example, the existing SDM630 recipe is version 1.1.4 while this commit's Lua table reports 1.1.1. Hosts and operators inspecting the loaded driver can consequently identify a different release from the artifact and history that were verified. Preserve the equality check and update promoted metadata alongside the package version.
AGENTS.md reference: AGENTS.md:L27-L29
Useful? React with 👍 / 👎.
…hem (#29) The driver release failed on both #27 and #28 with error: deye: changed artifact needs a higher version than 1.2.2 and it was right to. The channel refuses to publish changed bytes under a version it has already published; that immutability is what makes rollback mean anything. Nothing shipped, which is the correct outcome for a release that would have replaced 19 drivers' contents behind their own version numbers. So the 17 remaining replaced drivers move to a major version (pixii and sungrow already had). Major rather than patch because these are different implementations, not fixes, and an operator pinning a version needs to know that. bump_driver no longer rewrites the DRIVER table of a driver that is byte-identical to its FTW baseline. Doing so broke the byte-identity of 17 drivers, which cost them their test exemption and failed 13 checks -- and it spent the provenance guarantee on a field the tooling already accepts as differing. The manifest counts this repository's releases; the DRIVER table counts FTW's. Verified by building the signed publication locally against the currently published manifest: 80 drivers, 18 new, 19 bumped, zero version collisions. Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
1 participant
The mistake this fixes
This repository held 62 drivers of unknown provenance and treated them as the truth. FTW held 37 that have run on customer sites for months.
Every serious defect found while working here was in the catalog, never in FTW's drivers:
rated_Walways emitted as zeroThe wrong set was being repaired. So the drivers move rather than converge.
What changed
All 37 promoted verbatim — 19 replace a catalog driver of the same name, 18 are new. Byte-identical to
baselines/ftwon purpose: that is what makes provenance checkable and any later edit visible.Manifests are derived from what each driver already declares in its
DRIVERtable rather than retyped, so the two cannot drift.Three rules were invented rather than measured
DER vocabulary knew only
pv,battery,meter,v2x_charger. FTW also emitsev,heatpumpandvehicle— eight shipped drivers had no honest way to describe themselves.myuplinkemits only metrics and now says so with an empty list.The host-API profile called
modbus_writeandmillis"spellings this catalog grew on its own". They are FTW's real API names. It also omitted the entire transport layer —ws_open,tcp_open,mqtt_pub,persist_secret. The list now comes from FTW's Go binding.Canonical debt measured distance from an ideal that never existed. Deleted.
tools/host_api_check.pyreplaces it with the only question that predicts a crash: does a driver call a function no host provides? That is exactly what 35 drivers were doing while passing every test here.Two dialects, not a right and a wrong one
FTW and Blixt spell some functions differently. Both are real APIs of shipping hosts, and a host that wants the other's drivers adds aliases — FTW did so in about thirty lines. Converting drivers to one spelling was tried and abandoned: 196 lines across 36 field-proven drivers, changing what none of them does.
Two lineages, deliberately
A promoted driver's
DRIVERtable states FTW's id and version; the file name and manifest state this repository's. Forcing them equal would mean editing files kept unmodified on purpose, so the checks compare rather than demand.The blueprint
blueprint/BLUEPRINT.lua— a complete working driver for an imaginary inverter, written for people and agents alike. Every rule appears beside the code that follows it.It is verified rather than trusted.
tests/test_blueprint.pycompiles it, runs the sandbox and host-API checks, and polls it against the harness — including the test that matters most: failed reads per poll must settle to zero. A blueprint that would take a site offline teaches the mistake it demonstrates.docs/WRITING-A-DRIVER.mdgives the reasoning, each rule tied to the incident that produced it.Test suite
Catalog conventions are skipped for a driver byte-identical to its baseline, since FTW tests those in Go. Identity is by content, not a list that would rot — edit a promoted driver here and every check applies to it again.
Tests pinning the behaviour of replaced catalog drivers are removed; they tested code that no longer exists.
make check: 1924 passed, 80 drivers, sandbox and host-API clean.Known and recorded, not hidden
ferroamp_dc2_v2xcallsos.time()for an MQTT timestamp. FTW clearsosonly for control-v2 drivers, so it works today and would fail the day it is published through the signed channel. There is no host wall-clock to replace it with, and dropping the field would change a contract with whoever consumes that topic. Recorded as a bundled-only exception intools/check_sandbox.shrather than quietly passed.🤖 Generated with Claude Code