Skip to content

Draft: repository-owned YAML processor (Core 7.6+ conformance zero-fail)#37

Draft
Marius Storhaug (MariusStorhaug) wants to merge 24 commits into
mainfrom
yaml-conversion-foundation
Draft

Draft: repository-owned YAML processor (Core 7.6+ conformance zero-fail)#37
Marius Storhaug (MariusStorhaug) wants to merge 24 commits into
mainfrom
yaml-conversion-foundation

Conversation

@MariusStorhaug

@MariusStorhaug Marius Storhaug (MariusStorhaug) commented Jul 19, 2026

Copy link
Copy Markdown
Member

Summary

This draft PR delivers a dependency-free, repository-owned YAML processor with a PowerShell Core 7.6+ runtime contract and full green CI.

Implemented

  • removed all YamlDotNet dependency surfaces (assemblies, notices/licenses, loader assumptions, compatibility rewrite)
  • implemented an owned layered processor in source:
    • Unicode/c-printable input validation + reader context
    • directive scanning
    • context-aware block/flow syntax readers
    • iterative representation graph composer
    • core-schema scalar construction + PowerShell projection
    • iterative serializer + YAML emitter
  • fixed tag URI expansion to decode % escapes as UTF-8 (including multibyte), with malformed escape rejection
  • kept duplicate-key rejection as a load/composition policy (2JQS, X38W)
  • kept explicit JSON projection policy differences only for:
    • 565N (!!binary => byte[])
    • J7PZ (!!omap => OrderedDictionary)
  • updated Process-PSModule to v6.1.13 and migrated docs config to zensical (.github/zensical.toml)
  • added src/header.ps1 with:
    • #Requires -Version 7.6
    • #Requires -PSEdition Core
      so Build-PSModule emits artifact manifest/runtime metadata for Core 7.6+

Conformance accounting (full 402-input corpus)

Pinned corpus archive: yaml-test-suite-data-2022-01-17.zip
SHA-256: DCC1F037B13F6C3032D5190C447B6A6EF5560738A4C104F29B4243B0AB8F8029

  • Syntax: Pass 400 / PolicyDifference 2 / Fail 0 / NotApplicable 0
  • Event: Pass 308 / PolicyDifference 0 / Fail 0 / NotApplicable 94
  • JSON: Pass 277 / PolicyDifference 2 / Fail 0 / NotApplicable 123
  • out.yaml: Pass 241 / PolicyDifference 0 / Fail 0 / NotApplicable 161
  • Emit + round-trip: Pass 306 / PolicyDifference 0 / Fail 0 / NotApplicable 96

Validation

Local

  • pwsh -NoLogo -NoProfile -Command "Invoke-Pester -Path .\tests -Output Detailed"
    • 154 passed, 0 failed, 0 skipped
  • pwsh -NoLogo -NoProfile -Command "Invoke-ScriptAnalyzer -Path .\src -Recurse -Settings .\.github\linters\.powershell-psscriptanalyzer.psd1"
    • 0 findings

CI

  • Process-PSModule run: 30033823470
    • Build-Module, Build-Docs, Build-Site ✅
    • Lint-Repository + Lint-SourceCode (Windows/Linux/macOS) ✅
    • Test-SourceCode, Test-Module, Test-ModuleLocal (Windows/Linux/macOS) ✅
    • Get-TestResults, Get-CodeCoverage ✅
  • CodeQL run: 30033815695

Coverage gate is configured to PercentTarget: 25 in .github/PSModule.yml (current aggregate run reports 25.9%).

Draft status

PR remains draft for coordinator review.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add YAML 1.2 core-schema construction, safe tag handling, duplicate-key validation, aliases, resource limits, object projection, Test-Yaml, and pinned conformance fixtures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Normalize supported PowerShell data, emit a YAML 1.2-compatible subset, preserve repeated acyclic references, reject cycles and normalized duplicate keys, and cover packaging metadata.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Document command contracts, safety limits, pipeline behavior, supported data types, unknown-tag handling, and non-preserved YAML presentation details with working examples.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 19, 2026

Copy link
Copy Markdown

Super-linter summary

Language Validation result
CHECKOV Pass ✅
GITHUB_ACTIONS Pass ✅
GITLEAKS Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
MARKDOWN Pass ✅
NATURAL_LANGUAGE Pass ✅
POWERSHELL Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Pass ✅
TRIVY Pass ✅
YAML Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

Prefer the workflow-imported artifact so Pester coverage measures real module execution, exclude pinned YAML fixtures from repository config linting, and skip PlatyPS docs generation because it preloads a conflicting YamlDotNet assembly.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use yamllint's native ignore rules for vendored spec and invalid parser fixtures without relying on unsupported reusable-workflow environment settings.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Match absolute fixture paths passed by Super-Linter and satisfy repository analysis for the coverage test helper.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Align workflow pin and packaging assertions to Process-PSModule v6.1.13, and disable Build-Site until this repo migrates from mkdocs to zensical.toml required by the latest release.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replace legacy .github/mkdocs.yml with .github/zensical.toml and re-enable normal Build-Site execution under Process-PSModule v6.1.13.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove YamlDotNet artifacts and implement repository-owned layered YAML reader/composer/projector/emitter with iterative graph traversal, safety limits, and explicit tag handling.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add deterministic yaml-test-suite runner surfaces for syntax, event-structure, JSON construction, out.yaml comparison, and emitter round-trip accounting; update packaging and regression tests for PowerShell 7.6 Core-only contract and zensical docs config.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Mark .zip fixtures as binary in .gitattributes so the pinned yaml-test-suite archive is not altered by line-ending normalization.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@MariusStorhaug Marius Storhaug (MariusStorhaug) changed the title Add YAML 1.2 conversion foundation Draft: repository-owned YAML processor (Core 7.6 contract) Jul 23, 2026
Handle explicit keys with node properties followed by indentless block sequences, and add a regression test based on yaml-test-suite 6BFJ out.yaml shape.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add explicit-key indentless-sequence parsing for 6BFJ, and fix suite runner document-array argument passing so multi-document JSON/out.yaml comparisons evaluate all documents. Add regressions and update conformance assertions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve expanded tags in the representation graph for event comparisons and normalize yaml-test-suite event parsing/output so scalar escapes, style markers, flow markers, and anchor reuse are compared semantically instead of presentation text.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Read and write internal value boxes via raw PSObject properties so empty arrays and nested complex keys are preserved without pipeline-style flattening. Add a regression for nested empty-sequence keys and lock conformance accounting to the new zero-fail corpus counts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@MariusStorhaug Marius Storhaug (MariusStorhaug) changed the title Draft: repository-owned YAML processor (Core 7.6 contract) Draft: repository-owned YAML processor (Core 7.6+ conformance zero-fail) Jul 23, 2026
Decode %xx escapes as UTF-8 during tag-handle expansion, reject malformed/invalid UTF-8 escapes, and preserve unknown expanded tags in representation comparisons. Update conformance expectations so event surface has no policy differences and keep JSON policy cases explicitly enumerated.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Make test bootstrap deterministic by sourcing repository code unless an artifact manifest override is supplied, and make packaging artifact detection resolve from env/output paths so module-local CI does not bind to ambient gallery modules.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Inline percent-escape hex parsing in the tag URI decoder to satisfy source linting and recommit the yaml-test-suite release archive as binary bytes so the pinned SHA-256 matches in CI and module-local conformance runs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant