Skip to content

Enable strict source-aware cmdlet directive migrations - #3832

Open
Ramses Sanchez-Hernandez (ramsessanchez) wants to merge 6 commits into
powershell-v3from
ramsessanchez-directive-migration-framework
Open

Ramses Sanchez-Hernandez (ramsessanchez) wants to merge 6 commits into
powershell-v3from
ramsessanchez-directive-migration-framework

Conversation

@ramsessanchez

@ramsessanchez Ramses Sanchez-Hernandez (ramsessanchez) commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

No linked issue; this is the approved shared prerequisite for independent module directive-migration PRs.

Changes proposed in this pull request

  • Extend the existing inventory to all 44 configured modules, including modules without local markdown, inherited shared/default-formatting sources, and profile-local directives. Source-aware ledger identities retain the legacy module-local default. Strict completion requires reviewed source fingerprints, exact method/path/operation IDs, actual specification identities, explicit reviewed absences or source-based profile non-applicability, and complete operation-ID suppression mappings. Automatically matched candidates cannot pass completion. Parameter changes, cmdlet aliases, transforms, schema changes, formatting, and incomplete/no-effect selectors are classified separately.
  • Add minimal approved curated verb overrides, optional API-profile scoping, and explicit canonical module scoping to the existing configurator. Explicit curated names take precedence over derived names/suppressions; explicit curated suppressions still win. Regression tests cover precedence, defaults, approved verbs, profile/module isolation, unaffected methods/neighbors, and actual emitted output. Reconcile Compliance's stale pilot evidence: its historical nested dataSource routes are absent from both current specs. Preserve historical naming tests without claiming current generated coverage. AutoRest directives remain active; specs are unchanged.
  • Add a reusable strict dual-profile publication gate using the existing builder. Require full Kiota and wrapper generation, collision enforcement, compilation, Status OK, matching positive emitted/compiled cmdlet counts, non-fallback inputs, stable input fingerprints, and fresh expected sources/projects/locks/assemblies/manifests. Reject missing, failed, empty, skipped, stale, or shortcut evidence. Document the later module PR's three commits: configuration/mappings/tests, v1 sources, beta sources, with BOTH full gates passing before either source commit or publication.

Framework-only prerequisite based on b16f02829c, targeting powershell-v3. No module-generated trees are included. Missing-input exclusions and unrelated generator issues are unchanged. This does not mark any module's inherited directive inventory complete and does not authorize merges.

Validation

Commands ran from this isolated worktree. Evidence is retained outside the repository in the session artifact directory.

Command Outcome
.\tools\Test-WrapperDirectiveMigrationInventory.ps1 Passed: all 44 configured modules, source identities, classifications, strict pass/failure fixtures, legacy source identity, operation-ID mappings, profile isolation, CRLF/LF-portable spec fingerprints, and actual Compliance absences.
dotnet test .\tools\WrapperGenerator.Tests --configuration Release --nologo -v minimal Passed: 205 tests, 0 failed, 0 skipped. Existing NU1903 warning for Microsoft.OpenApi.YamlReader 3.7.0 remains; dependency remediation is outside this prerequisite.
.\tools\Test-WrapperPublicationGate.ps1 Passed: fresh pair accepted; missing profile, failed/NO-CMDLETS/skipped results, count mismatches, fallback/wrong spec identities, stale client/cmdlet/binary/manifest artifacts and forbidden shortcuts rejected. Includes real builder missing-spec preflight and JSON date handling.
.\tools\Test-WrapperPublication.ps1 -Module SchemaExtensions -Configuration Release -ResultPath 'C:\Users\ramsess\.copilot\session-state\259a08a8-ec49-4555-8a08-4e6a7249ef13\files\schemaextensions-publication.json' Passed: full Kiota + WrapperGenerator + Release compilation for both v1.0 and beta; each emitted/exported 7 cmdlets. Fresh artifacts and stable input fingerprint verified. Final profile build times: 29.2s / 32.3s. No GenerateOnly, SkipKiota or NoCollisionData.
.\tools\Compare-WrapperCmdletNames.ps1 -GeneratedPath .\src\SchemaExtensions\wrapper\v1.0\Cmdlets -OutLedger 'C:\Users\ramsess\.copilot\session-state\259a08a8-ec49-4555-8a08-4e6a7249ef13\files\schemaextensions-parity-v1.csv' Passed: 6/6 operation cmdlets match the oracle, plus one dispatcher.
.\tools\Compare-WrapperCmdletNames.ps1 -GeneratedPath .\src\SchemaExtensions\wrapper\beta\Cmdlets -OutLedger 'C:\Users\ramsess\.copilot\session-state\259a08a8-ec49-4555-8a08-4e6a7249ef13\files\schemaextensions-parity-beta.csv' Failed, pre-existing: six Mg-versus-MgBeta prefix mismatches, plus one dispatcher. All seven generated beta Cmdlet attributes were compared with the pristine base snapshot and are unchanged. Disclosed rather than converted into a pass; beta naming conventions are not changed here.
git --no-pager diff --check and staged diff check Passed. Pilot-generated trees restored to pristine originals and excluded from the commit.

The publication command invokes Build-WrapperModule.ps1 with explicit -ApiVersion v1.0 and -ApiVersion beta, -Configuration Release -StrictPublication, and separate JSON result paths. Pilot sources/binaries and per-profile logs/results are preserved in session artifacts, not this PR. Packaging, live Graph calls, and per-module migration completion are not claimed. Strict inventory still correctly rejects unreviewed inherited selectors; reviewed absence is not a substitute for missing module input.

Initial source-only compatibility evidence (historical)

The coordinator's cross-module regression comparison passed for all 38 available v1.0 modules against pristine b16f02829c: 12,371 generated *.g.cs files before and after, byte-for-byte identical, zero added/removed/changed files, and zero generation failures. This verifies default curated/derived precedence compatibility beyond the unit tests. This was WrapperGenerator source generation only, not full Kiota generation or module compilation, and does not substitute for any module's dual-profile publication gate.

The tested generator snapshot's SHA-256 is F1C23A896BFB74A82D1E0E6DB163B27F4C5D14D44933F316A71BB5B06E613FFB. Results and snapshot provenance are retained at C:\Users\ramsess\.copilot\session-state\e8b3909a-90be-4624-8e87-8f0e072ec770\files\framework-v1-regression\results.json and files\framework-generator-snapshot\snapshot-provenance.json in that same coordinator session. The prerequisite session independently checked all 38 result records and the snapshot assembly identity before adding this evidence.

Publication gate review follow-up

New commit 09892b62c3a7bb4ddfa614c0efeb3173e89cfa9f includes the actual Authentication project dependency, Authentication.Core, plus its imported Repo.props, in the input fingerprint. A shared separator-neutral predicate now excludes compiler bin/obj artifacts consistently in the generator, curated-config, runtime, authentication, Authentication.Core, and generated-client source scans.

  • .\tools\Test-WrapperPublicationGate.ps1 — Passed after the fixes. Focused tests prove that Core source/project and imported props changes invalidate the fingerprint; compiler output does not; Windows, Unix, and mixed-separator paths are filtered correctly; and stale compiler-generated client obj sources are not mistaken for stale generated API sources.
  • .\tools\Test-WrapperPublication.ps1 -Module SchemaExtensions -Configuration Release -ResultPath 'C:\Users\ramsess\.copilot\session-state\259a08a8-ec49-4555-8a08-4e6a7249ef13\files\schemaextensions-publication-review.json' — Passed after the fixes: fresh full Kiota + WrapperGenerator + Release compilation for v1.0 and beta, each 7 emitted / 7 compiled cmdlets, stable dependency-inclusive input fingerprint, and fresh required artifacts. Profile build times: 99.0s / 67.8s. This full pilot ran on Windows; the path-style fixture checks are not a claim of a full Linux build.

Pilot output was again preserved outside the repository and restored to pristine originals before the new commit. No module-generated tree changes or unrelated generator changes were introduced. The all-38 source-only comparison remains applicable: this follow-up changes publication tooling/tests/documentation, not generator naming or emission code.

Inventory identity and reviewed translation follow-up

New commit cdce7fc80d9a72c9ccb9e291bd38b0259a58d994 fixes the two module-discovered inventory gaps without changing C# generation, specifications, module configurations, or generated trees:

  • Decode actual plain/single-/double-quoted YAML path scalars, including doubled apostrophes in alternate-key and function paths. Keep operations attached to their own path; reject malformed keys, missing IDs, duplicate operations, unsupported inline operations and referenced path items rather than silently claiming absence. Preserve case-distinct identities.
  • Add optional per-profile sourceSpec: { specPath, specSha256, operations: [{ method, path, operationId }] } evidence to the existing ledger for operation-ID suppressions and the existing literal Graph[BWAFOR]Pre rename rules. Pin the canonical checked-in AutoRest counterpart, canonical UTF-8/LF hash, and the complete exact original selector candidate set. Existing operations pins exact current targets; no arbitrary replacement regex or new configurator is introduced. Reviewed source/current identities are mandatory. Reject stale hashes, missing/duplicate/unmatched originals or targets, arbitrary routes and false absence. Genuinely removed source routes require exact original absentOperations identities with per-entry evidence.
  • Verify equivalent HTTP methods and fixed route structure, normalizing only parameter names and the proven terminal microsoft.graph. spelling of a POST action or GET function when both pinned specs declare the same operation type. At this commit, type-cast aliases were not accepted; the narrowly verified follow-up below adds that separate case. Navigation, intermediate namespace segments, call names and argument syntax are not relaxed. Inventory accurately includes alternate-key operations that the generator does not currently emit; generator support is unchanged.
  • Document the exact schema and repository-built Authentication/Core module closure in standard module layout, selected through an isolated session-local PSModulePath, rather than an incompatible installed release. No builder expansion or global installation is needed.
Follow-up command Outcome
.\tools\Test-WrapperDirectiveSourceTranslation.ps1 Passed: quoted YAML paths and failure/neighbors; complete reviewed translations; stale/unreviewed/missing/duplicate/arbitrary mapping rejection; genuine removed-route evidence; false-absence rejection; parameter-name and narrowly proven qualified-call equivalence. Real corpus checks cover CrossDevice's two collection suppressions and all 11 inherited beta GraphBPre operations including three alternate-key operations; People's three beta analytics routes without activityStatistics leakage; DirectoryObjects getByIds; and BackupRestore's 46 v1/52 beta GraphFPre source operations plus quoted beta statistics function. ChangeNotifications-shaped action/function fixtures cover qualification and metadata failures.
.\tools\Test-WrapperDirectiveMigrationInventory.ps1 Passed again on the final implementation: all 44 configured modules, existing compatibility/strict fixtures, plus the integrated new source-translation suite. Final run: 348.2s.
git --no-pager diff --check and git --no-pager diff --cached --check Passed. Follow-up commit contains only inventory tooling, focused tests, and README changes.

This inventory-only follow-up does not rerun the C# suite or the full module build pair because neither C# nor publication/build tooling changed. The earlier 205/205 tests, dual-profile SchemaExtensions build evidence, fingerprint fixes, and all-38-module source-only compatibility evidence are retained, not recharacterized as new compilation results. Review-candidate JSON and test logs are preserved at C:\Users\ramsess\.copilot\session-state\259a08a8-ec49-4555-8a08-4e6a7249ef13\files\directive-translation-real-candidates.json, directive-source-translation.log, and directive-inventory-translation.log in that same directory. Those candidates do not claim any module migration is complete.

Final same-type cast and module-isolation follow-ups

Commit 3931c701537991ab229d63211bf92b8a1330baca adds only the proven terminal microsoft.graph.<Type> to graph.<Type> cast alias when both pinned specifications declare the exact same case-sensitive microsoft.graph.<Type> schema. HTTP method, type and fixed prefix remain identical; action/function operations cannot use this cast rule. Tests cover all 12 original CrossDevice GraphOPre candidates, including the 8 existing typed-cast counterparts, and reject changed method/type/case/namespace or missing schema evidence. No existing casts are relabeled absent. The ledger schema is unchanged.

Commit 2417a842daad327aa4f642660d8c354e11d5d7be isolates curated module-owned rules using the actual configured identity, not a collapsed client namespace or caller path:

  • Optional trailing Entry(..., ApiVersion: "beta", Module: "Devices.ServiceAnnouncement") and GeneratorConfig(..., Module: "Devices.ServiceAnnouncement"); standalone CLI --module Devices.ServiceAnnouncement. Every new module migration entry must specify its exact owner, including dispositions of inherited shared directives. Existing unscoped rules remain global for compatibility.
  • The generator embeds config\ModulesMapping.jsonc and rejects unknown, collapsed or incorrectly cased module identities. Both builder invocation paths and coverage tools pass the actual module. An applicable scoped rule without module context fails visibly.
  • Matching scoped noun/verb rules precede legacy global naming rules; same-scope registration order remains deterministic, explicit curated suppression still wins, and derived-data precedence is unchanged. Tests cover suppression and noun/verb/prefix renames at identical method/path across modules and profiles, defaults, missing context, neighbors, CLI and generated output.
  • The real 9 beta administrativeUnits route overlaps remain suppressible in Devices.ServiceAnnouncement without suppressing Identity.DirectoryManagement. The prerequisite's two historical Compliance beta entries now explicitly belong to Compliance. No other module migration or generated module tree is included.
  • Correct the Windows staging recipe to prepend the complete repository-built Authentication stage to the complete inherited PSModulePath, then assert actual fresh-child resolution. Truncating the inherited path can cause startup reinsertion of installed modules ahead of the stage. The coordinator withdrew the extra Authentication child-API request after actual-origin evidence; no new import option, global module change or fallback was added.
Final command / evidence Outcome
dotnet test .\tools\WrapperGenerator.Tests --configuration Release --no-restore --nologo -v minimal Passed: 235 tests, 0 failed, 0 skipped, on the final2417 C# snapshot. Generator inputs remain unchanged in the later test-only follow-up. Existing NU1903 warning remains.
.\tools\Test-WrapperDirectiveMigrationInventory.ps1 Passed after both follow-ups: all 44 modules and integrated strict source-translation/cast fixtures; 412.7s. Passed again after the migration-safe fixture fix below: 269.5s.
.\tools\Test-WrapperPublicationGate.ps1 Passed on the final generation/build implementation: dependency fingerprints, portable output exclusions and strict pair pass/failure fixtures.
.\tools\Test-WrapperPublication.ps1 -Module SchemaExtensions -Configuration Release -ResultPath 'C:\Users\ramsess\.copilot\session-state\259a08a8-ec49-4555-8a08-4e6a7249ef13\files\schemaextensions-publication-module-scope-final.json' Passed: full Kiota + WrapperGenerator + Release compilation, collision data enabled, explicit actual module context; 7 emitted / 7 compiled cmdlets per profile, 77.9s v1.0 / 86.2s beta, stable input fingerprint and fresh artifacts. No GenerateOnly, SkipKiota or NoCollisionData. An earlier run of the same final code also passed both full gates (144.0s / 104.0s); the repeated run retained the complete outputs.
.\tools\Compare-WrapperCmdletNames.ps1 -GeneratedPath '<session-files>\schemaextensions-module-scope-final\v1.0\Cmdlets' -OutLedger '<session-files>\schemaextensions-parity-module-scope-v1.csv' Passed: 6/6, plus one dispatcher.
.\tools\Compare-WrapperCmdletNames.ps1 -GeneratedPath '<session-files>\schemaextensions-module-scope-final\Beta\Cmdlets' -OutLedger '<session-files>\schemaextensions-parity-module-scope-beta.csv' Failed, pre-existing: the same 6 Mg/MgBeta prefix mismatches. All 7 complete Cmdlet attributes in each profile exactly match pristine originals. Not relabeled as a parity pass.
Fresh pwsh -NoProfile -NonInteractive -EncodedCommand compiled-import origin probes for both profiles Passed: each exports 7 cmdlets; actual Authentication and Core module/assembly paths and SHA-256 hashes identify this session's repository-built 2.38.1 stage. Complete inherited PSModulePath was preserved.
git --no-pager diff --check and git --no-pager diff --cached --check Passed after restoring all 48 original profile files byte-for-byte; worktree clean.
New 38-module source-only comparison with explicit actual module context Passed on final2417: all 38 available v1 modules; 12,371 before / 12,371 after *.g.cs files, byte-identical; zero added, removed, changed or failed. Every invocation passes its actual configured --module, independently recovered original client namespace and normal collision enforcement. Frozen generator SHA-256 before/after: 28C15FC72E696B7C8B505D80DA3F2BA9F0FA4F2DBB978B0760DB636BF7B23A0C. Source generation only, not Kiota generation or compilation/publication.

<session-files> in the parity commands is C:\Users\ramsess\.copilot\session-state\259a08a8-ec49-4555-8a08-4e6a7249ef13\files. Final logs/evidence there include wrapper-tests-module-scope.log, directive-inventory-module-scope.log, schemaextensions-publication-module-scope-final.json and its per-profile JSON/logs, schemaextensions-compiled-origin-v1.0.json, schemaextensions-compiled-origin-beta.json, and schemaextensions-name-baseline-module-scope.json. Complete final generated sources/binaries are preserved in schemaextensions-module-scope-final, outside the repository. The frozen generator and real input hashes are in framework-module-scope-snapshot\snapshot-provenance.json. The earlier 205-test and initial 38-module comparison results are historical and are not substituted for revalidation of changed C#. Each later module must still satisfy its own reviewed ledger and full dual-profile generation/compilation gates. No merge is authorized.

The coordinator's exact final source-comparison command completed with exit 0:

& 'C:\Users\ramsess\.copilot\session-state\e8b3909a-90be-4624-8e87-8f0e072ec770\files\Test-ModuleScopedV1Compatibility.ps1' `
    -RepositoryRoot 'C:\Users\ramsess\.copilot\repos\copilot-worktrees\msgraph-sdk-powershell\ramsessanchez-ubiquitous-invention' `
    -GeneratorDirectory 'C:\Users\ramsess\.copilot\session-state\e8b3909a-90be-4624-8e87-8f0e072ec770\files\framework-module-scope-snapshot' `
    -ResultDirectory 'C:\Users\ramsess\.copilot\session-state\e8b3909a-90be-4624-8e87-8f0e072ec770\files\module-scoped-v1-regression' `
    -FrameworkCommit '2417a842daad327aa4f642660d8c354e11d5d7be'

Results and provenance are retained in that result directory as results.json and provenance.json, with per-module logs/output alongside. Choose a new result directory to reproduce the command without overwriting evidence. The prerequisite session independently verified all 38 result records, the exact available configured roster and module contexts, each current spec hash, all zero-difference/exit-code fields and the frozen assembly identity. Its check summary is <session-files>\module-scoped-v1-regression-verified.json. This comparison is explicitly source-only, not module compilation or publication, and does not replace either profile's full gate.

Migration-safe inventory fixture follow-up

Final test-only commit e7a0cf5fb4ee027fbf7d8d1afa7cae64f58ad40b removes the assumptions that real Compliance must remain incomplete and real Calendar must retain unreviewed candidates. Successfully migrating either module must not break the shared regression suite. The existing synthetic Widget fixture now explicitly asserts Implementation = Candidate, still rejects strict unreviewed completion, and then passes after a complete reviewed ledger is supplied. Production roster, source/profile, neighbor and true Compliance absence checks remain.

  • .\tools\Test-WrapperDirectiveMigrationInventory.ps1 — Passed: all 44 modules and the complete integrated fixture suite, 269.5s. Log: <session-files>\directive-inventory-migration-safe.log.
  • This changes one test file only (1 insertion, 3 deletions), not C#, curated data, inventory runtime, builder, specifications or generated trees. Independently checked all 78 frozen generator inputs and the real pilot's full generation/build input fingerprint: unchanged (DA4DB50D463E90357D84895D0A8CFB42F607FEE2E2A3B142FE299B7DADFCBD71). Evidence: <session-files>\migration-safe-fixture-inputs-verified.json. The 235-test C# result, new 38-module source comparison and full dual-profile pilot therefore still cover the final generator/build implementation.

Other links

…ation

Inventory configured modules and effective shared/profile sources; validate reviewed rename and suppression mappings with portable source/spec identities. Add scoped curated verbs and precedence regressions, reconcile historical Compliance absences, and require fresh full generation plus compilation before module publication.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 259a08a8-ec49-4555-8a08-4e6a7249ef13
Microsoft Graph DevX Tooling and others added 5 commits October 20, 2026 17:05
…able paths

Include Authentication.Core and imported Repo.props inputs, exclude bin/obj with either separator on all gate source scans, and pin dependency changes and output exclusions with focused regression fixtures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 259a08a8-ec49-4555-8a08-4e6a7249ef13
…tions

Decode actual YAML path scalars without misattributing alternate-key operations. Pin complete original selector evidence and exact current targets, reject false absences and arbitrary translations, and verify only proven terminal Graph action/function qualification changes. Add real module regressions and document sourceSpec plus repository-built authentication prerequisites.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 259a08a8-ec49-4555-8a08-4e6a7249ef13
Preserve exact method/type identities while allowing the proven terminal microsoft.graph.Type to graph.Type alias only when both pinned specs declare the same schema. Cover all CrossDevice GraphOPre candidates, reject unsafe type/method/schema changes, and correct Windows child-module staging guidance.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 259a08a8-ec49-4555-8a08-4e6a7249ef13
Add optional canonical Entry.Module and GeneratorConfig.Module identities plus --module, pass actual module context through builders and coverage callers, and prefer scoped names over legacy globals without changing unscoped defaults. Fail ambiguous missing context, scope the Compliance pilot, and cover nine cross-module route overlaps, profiles, renames, precedence, CLI flow and generated output.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 259a08a8-ec49-4555-8a08-4e6a7249ef13
Assert unresolved candidate behavior against the synthetic Widget ledger instead of requiring production Compliance to remain incomplete or Calendar to retain candidates. Preserve strict rejection, reviewed completion, profile, neighbor and real absence coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 259a08a8-ec49-4555-8a08-4e6a7249ef13

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant