Skip to content

Ingest Li 2024 exact-pair formate data into 000068; add --from-file to cache_fulltext - #261

Merged
realmarcin merged 1 commit into
mainfrom
enrich-000068-formate-perturbation
Jul 28, 2026
Merged

Ingest Li 2024 exact-pair formate data into 000068; add --from-file to cache_fulltext#261
realmarcin merged 1 commit into
mainfrom
enrich-000068-formate-perturbation

Conversation

@realmarcin

Copy link
Copy Markdown
Contributor

Closes the retrieval half of #259 and enriches CommunityMech:000068.

The curator supplied the publisher PDF for Li et al. 2024 (doi:10.3390/w16243551, Water 16:3551) — the paper #259 could not retrieve, being absent from PubMed and Europe PMC with MDPI returning HTTP 403 to programmatic download.

Tooling

cache_fulltext.py gains --from-file <path>: a curator-supplied PDF/HTML/text file is appended under the existing open-access marker, for either a PMID or a DOI cache entry. pypdf is imported lazily with a message pointing at uv run --with pypdf, so it stays off the project dependency list. Writing is refused if extraction yields under 500 chars, so a failed parse can't silently produce an empty cache.

$ uv run --with pypdf python scripts/cache_fulltext.py doi:10.3390/w16243551 --from-file water-16-03551.pdf
[cached] 10.3390/w16243551: appended 66496 chars from water-16-03551.pdf -> DOI_10.3390_w16243551.md
$ ... (again)
[ok] 10.3390/w16243551: full text already cached

Curation

⚠️ The paper runs an anaerobic-sludge system alongside the coculture. Every result was traced to its system before use; only exact-coculture results are curated. Confirmed via: "we selected Syntrophobacter fumaroxidans and Methanobacterium formicicum to establish a co-culture system…"

  • New Exogenous Formate Dosage environmental factor: 5–10 mM raises propionate degradation efficiency (p < 0.01, days 5 and 7); 50 mM causes propionate accumulation at days 5–7.
  • The dose response is recorded as non-monotonic — at 30 mM propionate metabolism "was improved instead of inhibited in the later stage". A >=30 mM inhibits reading would be wrong for this coculture.
  • FDH1–4 downregulation under formate stress added to the Propionate Oxidation interaction. The authors' link from that to inhibited oxidation is curated PARTIAL, because they state it as what "may be the main reason" — association, not knockout-proven.
  • The partner-attribution discussion moves to PARTLY RESOLVED, recording both cautions plus the still-open H2-vs-formate apportionment question.

One snippet did not survive checking

The earlier causal-graph report offered "MMC metabolism of propionate was inhibited when the formate dosage reached 50 mM" as a verbatim quote. It appears nowhere in the paper. Replaced with the actual wording. This is the third instance in this thread where checking a generated snippet against real source text mattered.

Verification

  • just validate clean
  • snippet audit MATCH 4101 → 4107 — exactly the 6 new snippets, zero new mismatches, NOCONTENT unchanged
  • network-integrity unchanged (40)
  • just test 266 passed; ruff clean on the changed script

🤖 Generated with Claude Code

…o cache_fulltext (#259)

The curator supplied the publisher PDF for Li et al. 2024 (doi:10.3390/w16243551,
Water 16:3551), which #259 could not retrieve: absent from PubMed and Europe PMC,
and MDPI returns HTTP 403 to programmatic download.

Tooling: cache_fulltext.py gains `--from-file <path>`, accepting a
curator-supplied PDF/HTML/text file and appending it under the existing
open-access marker, for either a PMID or a DOI cache entry. pypdf is imported
lazily with a message pointing at `uv run --with pypdf`, so it stays off the
project dependency list. Refuses to write when extraction yields under 500
chars, so a failed parse cannot silently produce an empty cache.

Curation (000068): the paper runs an anaerobic-sludge system ALONGSIDE the
S. fumaroxidans-M. formicicum coculture, so each result was traced to its system
before use; only exact-coculture results are curated.
- New `Exogenous Formate Dosage` environmental factor: 5-10 mM raises propionate
  degradation efficiency, 50 mM causes propionate accumulation at days 5-7.
- The dose response is explicitly recorded as NON-monotonic: 30 mM recovers in
  the later stage, so ">=30 mM inhibits" would be wrong for this coculture.
- FDH1-4 downregulation under formate stress added to the Propionate Oxidation
  interaction; the authors' link from that to inhibited oxidation is curated
  PARTIAL because they state it as what "may be the main reason".
- The partner-attribution discussion is updated to PARTLY RESOLVED, recording
  both cautions and the still-open H2-vs-formate apportionment question.

One snippet from the earlier causal-graph report did not survive checking
against the real PDF ("MMC metabolism of propionate was inhibited when the
formate dosage reached 50 mM" appears nowhere in the paper) and was replaced
with the paper's actual wording.

Verification: validate clean; snippet audit MATCH 4101 -> 4107, exactly the 6
new snippets, no new mismatches; network-integrity unchanged; 266 tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@realmarcin
realmarcin merged commit fa78962 into main Jul 28, 2026
4 checks passed
@realmarcin
realmarcin deleted the enrich-000068-formate-perturbation branch July 28, 2026 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant