diff --git a/README.md b/README.md index 75a4428..3fec3ab 100644 --- a/README.md +++ b/README.md @@ -34,6 +34,14 @@ The openhound-github extension collects resources from Github organizations and edges for BloodHound. +### Enterprise SCIM and hybrid correlations + +When `SOURCES__GITHUB__COLLECT_ENTERPRISE_SCIM=true`, a token with enterprise SCIM access is used to collect both `/scim/v2/enterprises/{enterprise}/Users` and `/scim/v2/enterprises/{enterprise}/Groups`. The collector emits normalized `SCIM_Organization`, `SCIM_User`, and `SCIM_Group` nodes plus `SCIM_Contains`, `SCIM_MemberOf`, and `SCIM_Provisioned` relationships. Install the BloodHound SCIM extension alongside this extension to register the shared SCIM kinds. + +`SOURCES__GITHUB__EMIT_LEGACY_SCIM_CORRELATIONS=true` temporarily reproduces GitHound-style Okta-to-SCIM correlation relationships. It defaults to false because a dedicated hybrid correlator should own IdP-to-SCIM matching; GitHub remains authoritative for GitHub's SCIM resources and target-system provisioning relationships. + +Enterprise roles, including the built-in `members` role, are emitted through `GH_HasRole` and granular enterprise capability relationships. Only capability relationships with a confirmed privilege path are traversable; descriptive permissions such as `GH_WriteEnterpriseSso` remain non-traversable. + [![Python Version](https://img.shields.io/badge/Python-3.13-brightgreen.svg)](#about) ## Getting Started diff --git a/descriptions/edges/GH_ApprovesDeploymentTo.md b/descriptions/edges/GH_ApprovesDeploymentTo.md new file mode 100644 index 0000000..f67ad9d --- /dev/null +++ b/descriptions/edges/GH_ApprovesDeploymentTo.md @@ -0,0 +1,7 @@ +## General Information + +The non-traversable GH_ApprovesDeploymentTo edge represents that a user or team is configured as a required reviewer for a GitHub Environment. + +This edge is emitted from GH_User or GH_Team nodes to GH_Environment nodes when the environment includes a required reviewer protection rule. Required reviewers act as an approval gate before jobs referencing the environment can continue. + +The edge is non-traversable because being listed as a reviewer does not by itself grant deployment access, but it is important context for understanding how deployments are approved and which identities participate in environment protection workflows. diff --git a/descriptions/edges/GH_CanCreateEnvironment.md b/descriptions/edges/GH_CanCreateEnvironment.md new file mode 100644 index 0000000..10a4922 --- /dev/null +++ b/descriptions/edges/GH_CanCreateEnvironment.md @@ -0,0 +1,5 @@ +## General Information + +The traversable GH_CanCreateEnvironment edge is a computed edge indicating that a repository role can cause a new GitHub environment to be created in the repository. This is derived from the ability to create and modify runnable branches/workflows: if a workflow references an environment name that does not already exist, GitHub will create that environment automatically. + +This edge is useful for modeling OIDC and deployment scenarios where trust is tied to an environment name. An attacker who can create a new environment through workflow changes may be able to instantiate a trusted environment on demand, even if it was not previously configured. diff --git a/descriptions/edges/GH_CanDeployToEnvironment.md b/descriptions/edges/GH_CanDeployToEnvironment.md new file mode 100644 index 0000000..fe8f178 --- /dev/null +++ b/descriptions/edges/GH_CanDeployToEnvironment.md @@ -0,0 +1,7 @@ +## General Information + +The traversable GH_CanDeployToEnvironment edge represents the ability for a repository, branch, or repository role to satisfy the modeled deployment branch policy or administrator bypass condition for a GitHub Environment. + +This edge is computed from environment deployment branch policy, branch protection state, and administrator bypass behavior. For unrestricted environments, the repository and all contained branches may receive this edge. For protected-branch-only environments, only protected branches receive it unless the repository has no branch protection rules at all, in which case all branches can deploy. For custom branch policies, only branches matching a GH_EnvironmentBranchPolicy receive the edge. + +Required reviewers and wait timers are represented as environment metadata and GH_ApprovesDeploymentTo context rather than as additional GH_CanDeployToEnvironment edges. A branch edge therefore means the branch is eligible to target the environment, not that every runtime approval or timing gate has already been satisfied. diff --git a/descriptions/edges/GH_CanEditEnvironment.md b/descriptions/edges/GH_CanEditEnvironment.md new file mode 100644 index 0000000..9368442 --- /dev/null +++ b/descriptions/edges/GH_CanEditEnvironment.md @@ -0,0 +1,15 @@ +## General Information + +The traversable GH_CanEditEnvironment edge indicates that a repository role can modify the configuration of a GitHub environment. In the current model, this edge is emitted for the repository's built-in `admin` role to every environment contained in that repository. + +Editing an environment is security-relevant because environment configuration controls deployment protections such as required reviewers, self-review restrictions, wait timers, deployment branch policies, and the "allow administrators to bypass configured protection rules" setting. An attacker who can edit an environment may be able to weaken or remove those controls, making later deployment and secret access paths possible. + +This edge is distinct from GH_CanDeployToEnvironment: + +- **GH_CanEditEnvironment** means the role can manage the environment's settings. +- **GH_CanDeployToEnvironment** means the source satisfies the modeled deployment branch policy or administrator bypass condition for the environment. + +```mermaid +graph LR + role["GH_RepoRole admin"] ==>|GH_CanEditEnvironment| env["GH_Environment production"] +``` diff --git a/descriptions/edges/GH_Contains.md b/descriptions/edges/GH_Contains.md index 01de8b8..9522b0d 100644 --- a/descriptions/edges/GH_Contains.md +++ b/descriptions/edges/GH_Contains.md @@ -1,3 +1,3 @@ ## General Information -The non-traversable GH_Contains edge represents structural containment within the GitHub resource hierarchy. The organization serves as the top-level container for users, teams, repositories, roles, secrets, app installations, and personal access tokens. Repositories contain their own repo-level secrets, and environments contain environment-scoped secrets. This edge is created by the collector to establish the organizational hierarchy of GitHub resources and is not traversable because containment alone does not imply privilege escalation. +The non-traversable GH_Contains edge represents structural containment within the GitHub resource hierarchy. The organization serves as the top-level container for users, teams, repositories, roles, secrets, app installations, and personal access tokens. Repositories contain branches, workflows, branch protection rules, environments, and repo-level secrets and variables. Environments contain environment branch policies, environment-scoped secrets, and environment-scoped variables. This edge is created by the collector to establish the resource hierarchy and is not traversable because containment alone does not imply privilege escalation. diff --git a/descriptions/edges/GH_DeploysTo.md b/descriptions/edges/GH_DeploysTo.md index 4689fbe..5a286a3 100644 --- a/descriptions/edges/GH_DeploysTo.md +++ b/descriptions/edges/GH_DeploysTo.md @@ -1,3 +1,3 @@ ## General Information -The non-traversable GH_DeploysTo edge links a workflow job to the GitHub Environment it targets via the `environment:` key. This edge records which jobs deploy to which environments. Environments can gate deployments with protection rules (required reviewers, wait timers, deployment branch policies) and can expose environment-scoped secrets. +The non-traversable GH_DeploysTo edge links a workflow job to the GitHub Environment it targets via the `environment:` key. This edge records which jobs deploy to which environments. Environments can gate deployments with protection rules (required reviewers, wait timers, deployment branch policies) and can expose environment-scoped secrets and variables. diff --git a/descriptions/edges/GH_HasBranch.md b/descriptions/edges/GH_HasBranch.md index b65366a..fbd3269 100644 --- a/descriptions/edges/GH_HasBranch.md +++ b/descriptions/edges/GH_HasBranch.md @@ -1,3 +1,3 @@ ## General Information -The non-traversable GH_HasBranch edge represents the relationship between a repository and its branches. This edge links each collected branch to its parent repository. It is a structural edge that provides the foundation for understanding branch-level protections and access controls. While not traversable itself, it connects repositories to branches where traversable edges like GH_CanWriteBranch and GH_CanEditProtection model the effective access. +The non-traversable GH_Contains edge represents the relationship between a repository and its branches. This edge links each collected branch to its parent repository. It is a structural edge that provides the foundation for understanding branch-level protections and access controls. While not traversable itself, it connects repositories to branches where traversable edges like GH_CanWriteBranch and GH_CanEditProtection model the effective access. diff --git a/descriptions/edges/GH_HasEnvironment.md b/descriptions/edges/GH_HasEnvironment.md index 2f46635..f725769 100644 --- a/descriptions/edges/GH_HasEnvironment.md +++ b/descriptions/edges/GH_HasEnvironment.md @@ -1,3 +1,3 @@ ## General Information -The non-traversable GH_HasEnvironment edge represents the relationship between a repository or branch and its deployment environments. This edge links environments to the repositories that define them and to the branches that are allowed to deploy to them (via deployment branch policies). Environments are security-relevant because they can gate access to secrets and cloud credentials, and their deployment branch policies control which branches can trigger deployments. +Environment containment is modeled with the non-traversable GH_Contains edge from a repository to each deployment environment it defines. Branch deployment eligibility is modeled separately with GH_CanDeployToEnvironment and GH_MatchesEnvironmentPolicy rather than a containment edge. Environments are security-relevant because they can gate access to secrets and cloud credentials, and their deployment branch policies control which branches can trigger deployments. diff --git a/descriptions/edges/GH_HasSecret.md b/descriptions/edges/GH_HasSecret.md index 7bf1cac..6899dc3 100644 --- a/descriptions/edges/GH_HasSecret.md +++ b/descriptions/edges/GH_HasSecret.md @@ -1,3 +1,3 @@ ## General Information -The traversable GH_HasSecret edge represents the relationship between a repository or environment and the secrets accessible within that context. This edge shows which secrets are available in which scopes. Repositories can have access to both organization-level secrets (scoped to selected repositories) and repository-level secrets, while environments contain their own environment-scoped secrets. This edge is traversable because any principal that can push code to a repository (via GH_CanWriteBranch or GH_CanCreateBranch) can write a workflow that exfiltrates the secret values at runtime, making this a meaningful link in attack path analysis. +The traversable GH_HasSecret edge represents the relationship between a repository or environment and the secrets accessible within that context. This edge shows which secrets are available in which scopes. Repositories can have access to both organization-level secrets (scoped to selected repositories) and repository-level secrets, while environments expose their own environment-scoped secrets to jobs that target them. This edge is traversable because any principal that can execute a workflow in the relevant context may be able to exfiltrate secret values at runtime, making this a meaningful link in attack path analysis. diff --git a/descriptions/edges/GH_HasVariable.md b/descriptions/edges/GH_HasVariable.md index bf7061f..8054ec5 100644 --- a/descriptions/edges/GH_HasVariable.md +++ b/descriptions/edges/GH_HasVariable.md @@ -1,3 +1,3 @@ ## General Information -The traversable GH_HasVariable edge represents the relationship between a repository and the variables accessible within that context. This edge shows which variables are available in which scopes. Repositories can have access to both organization-level variables (scoped by visibility to all, private, or selected repositories) and repository-level variables defined directly on the repo. This edge is traversable because any principal that can push code to a repository (via GH_CanWriteBranch or GH_CanCreateBranch) can write a workflow that reads variable values at runtime, and variables may contain configuration data useful for lateral movement such as deployment URLs, service names, or environment identifiers. +The traversable GH_HasVariable edge represents the relationship between a repository or environment and the variables accessible within that context. This edge shows which variables are available in which scopes. Repositories can have access to both organization-level variables (scoped by visibility to all, private, or selected repositories) and repository-level variables defined directly on the repo, while environments expose their own environment-scoped variables to jobs that target them. This edge is traversable because any principal that can execute a workflow in the relevant context may be able to read variable values at runtime, and variables may contain configuration data useful for lateral movement such as deployment URLs, service names, or environment identifiers. diff --git a/descriptions/edges/GH_MatchesEnvironmentPolicy.md b/descriptions/edges/GH_MatchesEnvironmentPolicy.md new file mode 100644 index 0000000..bfb8e88 --- /dev/null +++ b/descriptions/edges/GH_MatchesEnvironmentPolicy.md @@ -0,0 +1,7 @@ +## General Information + +The non-traversable GH_MatchesEnvironmentPolicy edge links a branch to a GitHub Environment deployment branch policy that it satisfies. + +This edge is emitted when a branch name matches the pattern defined by a GH_EnvironmentBranchPolicy node, such as `main`, `release/*`, or `release/**/*`. The edge is structural rather than directly traversable because matching a policy alone does not guarantee deployment access; the environment may still require protected branches, reviewers, wait timers, or other controls. + +GH_MatchesEnvironmentPolicy is primarily used as supporting evidence for computed GH_CanDeployToEnvironment edges. diff --git a/descriptions/edges/GH_UsesSecret.md b/descriptions/edges/GH_UsesSecret.md index 2a8b875..50375b0 100644 --- a/descriptions/edges/GH_UsesSecret.md +++ b/descriptions/edges/GH_UsesSecret.md @@ -1,15 +1,16 @@ ## General Information -The traversable GH_UsesSecret edge links a workflow step to the secret it references via a `${{ secrets.NAME }}` expression. This edge reveals which secrets a step can access at runtime, enabling analysts to trace the blast radius of a compromised workflow. +The non-traversable GH_UsesSecret edge links a workflow job or step to the secret it references via a `${{ secrets.NAME }}` expression. This edge reveals which secrets a workflow component can access at runtime, enabling analysts to trace the blast radius of a compromised workflow. ### Matching strategy -Edges use `match_by: property` with two matchers to disambiguate between secrets with the same name across repositories: +Edges use `match_by: property` with scope-specific matchers to disambiguate between secrets with the same name across repositories and environments: - **GH_RepoSecret** is matched by `name` + `repository_id` (the GitHub node_id of the repository). - **GH_OrgSecret** is matched by `name` + `environmentid` (the node_id of the organization, which acts as the org-level secret scope). +- **GH_EnvironmentSecret** is matched by `name` + `deployment_environment_name` + `repository_id` when the parent job targets a concrete environment name. -This means one `${{ secrets.MY_SECRET }}` expression in a workflow can produce up to two GH_UsesSecret edges — one to the repo-level secret and one to the org-level secret — reflecting that either could supply the value at runtime depending on scope precedence. +This means one `${{ secrets.MY_SECRET }}` expression in a workflow can produce edges to repo-level, org-level, and environment-level secrets that share the same name in the applicable scopes. The environment-level edge is only emitted when the workflow job references a literal environment name rather than a dynamic expression. ### Context property diff --git a/descriptions/edges/GH_UsesVariable.md b/descriptions/edges/GH_UsesVariable.md index 746553d..55d16a1 100644 --- a/descriptions/edges/GH_UsesVariable.md +++ b/descriptions/edges/GH_UsesVariable.md @@ -1,15 +1,16 @@ ## General Information -The non-traversable GH_UsesVariable edge links a workflow step to the variable it references via a `${{ vars.NAME }}` expression. This edge maps variable consumption within workflows. Unlike secrets, variable values are readable via the API, making them lower sensitivity — but they can still influence workflow behavior (e.g., controlling target environments or feature flags). +The non-traversable GH_UsesVariable edge links a workflow job or step to the variable it references via a `${{ vars.NAME }}` expression. This edge maps variable consumption within workflows. Unlike secrets, variable values are readable via the API, making them lower sensitivity — but they can still influence workflow behavior (e.g., controlling target environments or feature flags). ### Matching strategy -Edges use `match_by: property` with two matchers to disambiguate between variables with the same name across repositories: +Edges use `match_by: property` with scope-specific matchers to disambiguate between variables with the same name across repositories and environments: - **GH_RepoVariable** is matched by `name` + `repository_id` (the GitHub node_id of the repository). - **GH_OrgVariable** is matched by `name` + `environmentid` (the node_id of the organization, which acts as the org-level variable scope). +- **GH_EnvironmentVariable** is matched by `name` + `deployment_environment_name` + `repository_id` when the parent job targets a concrete environment name. -This means one `${{ vars.MY_VAR }}` expression can produce up to two GH_UsesVariable edges — one to the repo-level variable and one to the org-level variable. +This means one `${{ vars.MY_VAR }}` expression can produce edges to repo-level, org-level, and environment-level variables that share the same name in the applicable scopes. The environment-level edge is only emitted when the workflow job references a literal environment name rather than a dynamic expression. ### Context property diff --git a/descriptions/nodes/GH_Environment.md b/descriptions/nodes/GH_Environment.md index 2e335a0..bdd3d45 100644 --- a/descriptions/nodes/GH_Environment.md +++ b/descriptions/nodes/GH_Environment.md @@ -1,3 +1,5 @@ ## Description -Represents a GitHub Actions deployment environment configured on a repository. Environments can have protection rules including required reviewers, wait timers, and deployment branch policies. When custom branch policies are configured, the environment is connected to specific branches; otherwise, it is connected directly to the repository. +Represents a GitHub Actions deployment environment configured on a repository. Environments can have protection rules including required reviewers, wait timers, administrator bypass behavior, and deployment branch policies. + +Repositories always contain their environments. When custom branch policies are configured, the environment also contains one or more GH_EnvironmentBranchPolicy nodes that describe which branches are allowed to deploy. Environment-scoped secrets and variables are modeled as child nodes of the environment and become available to workflow jobs that reference it. diff --git a/descriptions/nodes/GH_EnvironmentBranchPolicy.md b/descriptions/nodes/GH_EnvironmentBranchPolicy.md new file mode 100644 index 0000000..d99f699 --- /dev/null +++ b/descriptions/nodes/GH_EnvironmentBranchPolicy.md @@ -0,0 +1,5 @@ +## Description + +Represents a deployment branch policy attached to a GitHub Environment. These policies define which branches or branch patterns are allowed to deploy to the environment, such as `main`, `release/*`, or `release/**/*`. + +Environment branch policies are modeled as their own nodes so analysts can distinguish between the environment itself and the matching rules that govern deployment eligibility. diff --git a/descriptions/nodes/GH_EnvironmentSecret.md b/descriptions/nodes/GH_EnvironmentSecret.md index 994cb8e..b3fca68 100644 --- a/descriptions/nodes/GH_EnvironmentSecret.md +++ b/descriptions/nodes/GH_EnvironmentSecret.md @@ -1,3 +1,5 @@ ## Description Represents an environment-level GitHub Actions secret. These secrets are scoped to a specific deployment environment and are only available to workflow jobs that reference that environment. + +The containing environment is linked to the secret with GH_Contains and GH_HasSecret edges. Workflow steps that reference the secret by name receive GH_UsesSecret edges when their job targets the same environment. diff --git a/descriptions/nodes/GH_EnvironmentVariable.md b/descriptions/nodes/GH_EnvironmentVariable.md index 7d60e2e..898760b 100644 --- a/descriptions/nodes/GH_EnvironmentVariable.md +++ b/descriptions/nodes/GH_EnvironmentVariable.md @@ -1,3 +1,5 @@ ## Description Represents an environment-level GitHub Actions variable. These variables are scoped to a specific deployment environment and are only available to workflow jobs that reference that environment. Unlike secrets, variable values are readable via the API. + +The containing environment is linked to the variable with GH_Contains and GH_HasVariable edges. Workflow steps that reference the variable by name receive GH_UsesVariable edges when their job targets the same environment. diff --git a/extension/saved_searches/environments-admin-bypass.json b/extension/saved_searches/environments-admin-bypass.json index 08447e4..4868040 100644 --- a/extension/saved_searches/environments-admin-bypass.json +++ b/extension/saved_searches/environments-admin-bypass.json @@ -1,5 +1,5 @@ { "name": "GitHub: Environments Where Admins Can Bypass Protections", - "query": "MATCH p=(:GH_Repository)-[:GH_HasEnvironment]->(env:GH_Environment {can_admins_bypass: true})\nRETURN p\nLIMIT 1000", + "query": "MATCH p=(:GH_Repository)-[:GH_Contains]->(env:GH_Environment {can_admins_bypass: true})\nRETURN p\nLIMIT 1000", "description": "Finds deployment environments where administrators can bypass protection rules such as required reviewers and wait timers. Admins can deploy to these environments without any approval." } diff --git a/extension/saved_searches/repository-workflows.json b/extension/saved_searches/repository-workflows.json index bcaed4f..c349625 100644 --- a/extension/saved_searches/repository-workflows.json +++ b/extension/saved_searches/repository-workflows.json @@ -1,5 +1,5 @@ { "name": "GitHub: Repository Workflows", - "query": "MATCH p=(:GH_Repository)-[:GH_HasWorkflow]->(:GH_Workflow)\nRETURN p\nLIMIT 1000", + "query": "MATCH p=(:GH_Repository)-[:GH_Contains]->(:GH_Workflow)\nRETURN p\nLIMIT 1000", "description": "Returns all repository workflows" } diff --git a/extension/saved_searches/unprotected-branches.json b/extension/saved_searches/unprotected-branches.json index b5ace62..6db0b41 100644 --- a/extension/saved_searches/unprotected-branches.json +++ b/extension/saved_searches/unprotected-branches.json @@ -1,5 +1,5 @@ { "name": "GitHub: Unprotected Branches", - "query": "MATCH p=(repo:GH_Repository)-[:GH_HasBranch]-(:GH_Branch {protected: false})\nRETURN p\nLIMIT 1000", + "query": "MATCH p=(repo:GH_Repository)-[:GH_Contains]-(:GH_Branch {protected: false})\nRETURN p\nLIMIT 1000", "description": "Returns all unprotected branches in repositories." } diff --git a/extension/saved_searches/unprotected-default-branch-with-workflow.json b/extension/saved_searches/unprotected-default-branch-with-workflow.json index 70de2fb..7e566dc 100644 --- a/extension/saved_searches/unprotected-default-branch-with-workflow.json +++ b/extension/saved_searches/unprotected-default-branch-with-workflow.json @@ -1,5 +1,5 @@ { "name": "GitHub: Repositories with Workflows and Unprotected Default Branch", - "query": "MATCH p=(repo:GH_Repository)-[:GH_HasWorkflow]->(:GH_Workflow)\nMATCH p1=(repo)-[:GH_HasBranch]->(branch:GH_Branch)\nWHERE repo.default_branch = branch.short_name\nAND branch.protected = false\nRETURN p1\nLIMIT 1000", + "query": "MATCH p=(repo:GH_Repository)-[:GH_Contains]->(:GH_Workflow)\nMATCH p1=(repo)-[:GH_Contains]->(branch:GH_Branch)\nWHERE repo.default_branch = branch.short_name\nAND branch.protected = false\nRETURN p1\nLIMIT 1000", "description": "Returns all repositories that have GitHub Actions workflows and an unprotected default branch. This means that users with GH_WriteRepoContents to the Repository can overwrite or change the workflow." } diff --git a/extension/saved_searches/unprotected-default-branches.json b/extension/saved_searches/unprotected-default-branches.json index b9add1b..7defa37 100644 --- a/extension/saved_searches/unprotected-default-branches.json +++ b/extension/saved_searches/unprotected-default-branches.json @@ -1,5 +1,5 @@ { "name": "GitHub: Unprotected Default Branches", - "query": "MATCH p=(repo:GH_Repository)-[:GH_HasBranch]-(branch:GH_Branch {protected: false})\nWHERE repo.default_branch = branch.short_name\nRETURN p\nLIMIT 1000", + "query": "MATCH p=(repo:GH_Repository)-[:GH_Contains]-(branch:GH_Branch {protected: false})\nWHERE repo.default_branch = branch.short_name\nRETURN p\nLIMIT 1000", "description": "Returns all default branches in repositories that are not protected." } diff --git a/extension/schema.json b/extension/schema.json index 891f495..e0e1dfc 100644 --- a/extension/schema.json +++ b/extension/schema.json @@ -142,6 +142,14 @@ "icon": "leaf", "color": "#D5F2C2" }, + { + "name": "GH_EnvironmentBranchPolicy", + "display_name": "GitHub Environment Branch Policy", + "description": "A deployment branch policy attached to a GitHub environment, such as an exact branch name or wildcard pattern like release/*", + "is_display_kind": true, + "icon": "code-branch", + "color": "#B7E3A1" + }, { "name": "GH_OrgSecret", "display_name": "GitHub Org Secret", @@ -277,6 +285,121 @@ "description": "Container relationship for organizational hierarchy (org contains secrets/variables, repo contains secrets/variables, environment contains secrets/variables)", "is_traversable": false }, + { + "name": "GH_CreateEnterpriseOrganizations", + "description": "[Enterprise] Enterprise role can create organizations", + "is_traversable": false + }, + { + "name": "GH_EditEnterpriseCustomPropertiesForOrganizations", + "description": "[Enterprise] Enterprise role can edit custom properties for organizations", + "is_traversable": false + }, + { + "name": "GH_ManageEnterpriseAdmins", + "description": "[Enterprise] Enterprise role can manage enterprise administrators", + "is_traversable": true + }, + { + "name": "GH_ManageEnterpriseIdentityProvider", + "description": "[Enterprise] Enterprise role can manage the enterprise identity provider", + "is_traversable": false + }, + { + "name": "GH_ManageEnterpriseMembers", + "description": "[Enterprise] Enterprise role can manage enterprise members", + "is_traversable": true + }, + { + "name": "GH_ManageEnterpriseOrganizationAdmins", + "description": "[Enterprise] Enterprise role can manage organization administrators", + "is_traversable": true + }, + { + "name": "GH_ManageEnterpriseOrganizations", + "description": "[Enterprise] Enterprise role can manage organizations", + "is_traversable": false + }, + { + "name": "GH_ManageEnterpriseReferrals", + "description": "[Enterprise] Enterprise role can manage referrals", + "is_traversable": false + }, + { + "name": "GH_ManageEnterpriseTeams", + "description": "[Enterprise] Enterprise role can manage enterprise teams", + "is_traversable": false + }, + { + "name": "GH_ReadEnterpriseAuditLog", + "description": "[Enterprise] Enterprise role can read the audit log", + "is_traversable": false + }, + { + "name": "GH_ReadEnterpriseDomainVerification", + "description": "[Enterprise] Enterprise role can read domain verification data", + "is_traversable": false + }, + { + "name": "GH_ReadEnterpriseMembers", + "description": "[Enterprise] Enterprise role can read enterprise members", + "is_traversable": false + }, + { + "name": "GH_ReadEnterpriseOrgProjects", + "description": "[Enterprise] Enterprise role can read organization projects", + "is_traversable": false + }, + { + "name": "GH_ReadEnterpriseOrganizationAdmin", + "description": "[Enterprise] Enterprise role can read organization administration data", + "is_traversable": false + }, + { + "name": "GH_SetEnterpriseInteractionLimits", + "description": "[Enterprise] Enterprise role can set interaction limits", + "is_traversable": false + }, + { + "name": "GH_ViewEnterpriseActionsUsageMetrics", + "description": "[Enterprise] Enterprise role can view Actions usage metrics", + "is_traversable": false + }, + { + "name": "GH_ViewEnterpriseBilling", + "description": "[Enterprise] Enterprise role can view billing data", + "is_traversable": false + }, + { + "name": "GH_ViewEnterpriseSecretScanningAlerts", + "description": "[Enterprise] Enterprise role can view secret-scanning alerts", + "is_traversable": false + }, + { + "name": "GH_WriteEnterpriseActionsPolicies", + "description": "[Enterprise] Enterprise role can write Actions policies", + "is_traversable": false + }, + { + "name": "GH_WriteEnterpriseBilling", + "description": "[Enterprise] Enterprise role can write billing settings", + "is_traversable": false + }, + { + "name": "GH_WriteEnterprisePersonalAccessTokenPolicies", + "description": "[Enterprise] Enterprise role can write personal access token policies", + "is_traversable": false + }, + { + "name": "GH_WriteEnterpriseSso", + "description": "[Enterprise] Enterprise role can write SSO settings", + "is_traversable": false + }, + { + "name": "GH_WriteEnterpriseTeamMembers", + "description": "[Enterprise] Enterprise role can write enterprise team membership", + "is_traversable": false + }, { "name": "GH_Owns", "description": "Organization owns a repository", @@ -697,11 +820,6 @@ "description": "User or team is allowed to push to branches protected by this rule", "is_traversable": false }, - { - "name": "GH_HasBranch", - "description": "Repository has this branch", - "is_traversable": false - }, { "name": "GH_CanEditProtection", "description": "[Repository - Computed] Repo role can modify or remove branch protection rules for the repository/branch (computed from GH_EditRepoProtections + GH_ProtectedBy)", @@ -717,16 +835,6 @@ "description": "Branch protection rule protects this branch", "is_traversable": false }, - { - "name": "GH_HasWorkflow", - "description": "Repository has this workflow", - "is_traversable": false - }, - { - "name": "GH_HasEnvironment", - "description": "Repository or branch has/can deploy to this environment", - "is_traversable": false - }, { "name": "GH_HasSecret", "description": "Repository or environment has access to this secret", @@ -734,7 +842,7 @@ }, { "name": "GH_HasVariable", - "description": "Repository has access to this variable (org-level or repo-level)", + "description": "Repository or environment has access to this variable", "is_traversable": true }, { @@ -787,6 +895,16 @@ "description": "[Repository - Computed] Role can create new branches in this repository (unprotected branches that bypass the merge gate)", "is_traversable": true }, + { + "name": "GH_CanCreateEnvironment", + "description": "Repo role can create new GitHub environments in this repository by editing a workflow that references a nonexistent environment name", + "is_traversable": true + }, + { + "name": "GH_CanEditEnvironment", + "description": "Repo admin role can edit the configuration of this GitHub environment", + "is_traversable": true + }, { "name": "GH_CanAssumeIdentity", "description": "Repository can assume this cloud identity via OIDC federation (Azure workload identity or AWS IAM role)", @@ -818,28 +936,33 @@ "is_traversable": false }, { - "name": "GH_HasJob", - "description": "[Workflow] Workflow contains this job — GH_Workflow → GH_WorkflowJob", + "name": "GH_HasMember", + "description": "Enterprise or organization has this user as a member", "is_traversable": false }, { - "name": "GH_HasMember", - "description": "Enterprise or organization has this user as a member", + "name": "GH_UsesSecret", + "description": "[Workflow] Job or step references a secret by name — GH_WorkflowJob / GH_WorkflowStep → GH_RepoSecret / GH_OrgSecret / GH_EnvironmentSecret (scope match)", "is_traversable": false }, { - "name": "GH_HasStep", - "description": "[Workflow] Job contains this step — GH_WorkflowJob → GH_WorkflowStep", + "name": "GH_UsesVariable", + "description": "[Workflow] Job or step references a variable by name — GH_WorkflowJob / GH_WorkflowStep → GH_RepoVariable / GH_OrgVariable / GH_EnvironmentVariable (scope match)", "is_traversable": false }, { - "name": "GH_UsesSecret", - "description": "[Workflow] Step references a secret by name — GH_WorkflowStep → GH_RepoSecret / GH_OrgSecret (name match)", + "name": "GH_CanDeployToEnvironment", + "description": "[Computed] Repository, branch, or repo role can deploy to this GitHub environment after evaluating deployment branch policy and admin bypass behavior", + "is_traversable": true + }, + { + "name": "GH_MatchesEnvironmentPolicy", + "description": "Branch matches this environment deployment branch policy", "is_traversable": false }, { - "name": "GH_UsesVariable", - "description": "[Workflow] Step references a variable by name — GH_WorkflowStep → GH_RepoVariable / GH_OrgVariable (name match)", + "name": "GH_ApprovesDeploymentTo", + "description": "User or team is configured as a required reviewer for this environment", "is_traversable": false } ], @@ -848,11 +971,7 @@ "environment_kind": "GH_Organization", "source_kind": "GitHub", "principal_kinds": [ - "GH_User", - "GH_Team", - "GH_Repository", - "GH_Branch", - "GH_Environment" + "GH_User" ] } ], diff --git a/extension.yaml b/src/openhound_github/extension.yaml similarity index 100% rename from extension.yaml rename to src/openhound_github/extension.yaml diff --git a/src/openhound_github/graphql.py b/src/openhound_github/graphql.py index a485f57..11b410b 100644 --- a/src/openhound_github/graphql.py +++ b/src/openhound_github/graphql.py @@ -118,37 +118,34 @@ ENTERPRISE_SAML_QUERY = """ query EnterpriseSAML($slug: String!, $count: Int = 100, $after: String = null) { enterprise(slug: $slug) { - id - name slug ownerInfo { samlIdentityProvider { - id - issuer - ssoUrl - digestMethod - signatureMethod - idpCertificate externalIdentities(first: $count, after: $after) { totalCount nodes { guid id samlIdentity { + attributes { + metadata + name + value + } familyName givenName nameId username } scimIdentity { - username - givenName - familyName emails { - value primary type + value } + familyName + givenName + username } user { id @@ -166,6 +163,26 @@ } """ +ENTERPRISE_SAML_PROVIDER_QUERY = """ +query EnterpriseSAMLProvider($slug: String!) { + enterprise(slug: $slug) { + id + name + slug + ownerInfo { + samlIdentityProvider { + id + issuer + ssoUrl + digestMethod + signatureMethod + idpCertificate + } + } + } +} +""" + TEAMS_QUERY = """ query Teams($login: String!, $count: Int!, $after: String) { organization(login: $login) { @@ -307,56 +324,18 @@ """ SAML_QUERY = """ -query SAML($login: String!, $count: Int = 100, $after: String = null) { +query SAMLProvider($login: String!) { organization(login: $login) { id name login samlIdentityProvider { - digestMethod - externalIdentities(first: $count, after: $after) { - nodes { - guid - id - samlIdentity { - attributes { - metadata - name - value - } - familyName - givenName - groups - nameId - username - } - scimIdentity { - emails { - primary - type - value - } - familyName - givenName - groups - username - } - user { - id - login - } - } - pageInfo { - endCursor - hasNextPage - } - totalCount - } id - idpCertificate issuer - signatureMethod ssoUrl + digestMethod + signatureMethod + idpCertificate } } } @@ -369,7 +348,6 @@ name login samlIdentityProvider { - digestMethod externalIdentities(first: $count, after: $after) { nodes { guid @@ -382,7 +360,6 @@ } familyName givenName - groups nameId username } @@ -394,7 +371,6 @@ } familyName givenName - groups username } user { @@ -408,11 +384,6 @@ } totalCount } - id - idpCertificate - issuer - signatureMethod - ssoUrl } } } diff --git a/src/openhound_github/helpers.py b/src/openhound_github/helpers.py index 78a7c4b..36bd036 100644 --- a/src/openhound_github/helpers.py +++ b/src/openhound_github/helpers.py @@ -4,15 +4,12 @@ from dlt.common import jsonpath from dlt.sources.helpers import requests +from dlt.sources.helpers.rest_client.auth import AuthConfigBase from dlt.sources.helpers.rest_client.paginators import ( JSONResponseCursorPaginator, ) from requests import Request -from openhound_github.auth import ( - GitHubAppInstallationAuth, -) - logger = logging.getLogger(__name__) @@ -154,7 +151,7 @@ def _has_graphql_errors(response: requests.Response) -> bool: return isinstance(response_data, dict) and bool(response_data.get("errors")) -def github_retry_policy(auth: GitHubAppInstallationAuth): +def github_retry_policy(auth: AuthConfigBase): def retry_policy( response: Optional[requests.Response], exception: Optional[BaseException] ) -> bool: diff --git a/src/openhound_github/kinds/edges.py b/src/openhound_github/kinds/edges.py index 4423f76..17baeda 100644 --- a/src/openhound_github/kinds/edges.py +++ b/src/openhound_github/kinds/edges.py @@ -17,6 +17,8 @@ CAN_ACCESS = "GH_CanAccess" CAN_USE_RUNNER = "GH_CanUseRunner" CAN_CREATE_BRANCH = "GH_CanCreateBranch" +CAN_CREATE_ENVIRONMENT = "GH_CanCreateEnvironment" +CAN_EDIT_ENVIRONMENT = "GH_CanEditEnvironment" CAN_EDIT_PROTECTION = "GH_CanEditProtection" CAN_WRITE_BRANCH = "GH_CanWriteBranch" READ_REPO_CONTENTS = "GH_ReadRepoContents" @@ -35,6 +37,42 @@ MAPS_TO_USER = "GH_MapsToUser" SYNCED_TO_GH_USER = "GH_SyncedTo" +# Normalized OpenGraph-SCIM edges +SCIM_CONTAINS = "SCIM_Contains" +SCIM_MEMBER_OF = "SCIM_MemberOf" +SCIM_HAS_ROLE = "SCIM_HasRole" +SCIM_MANAGER_OF = "SCIM_ManagerOf" +SCIM_PROVISIONED = "SCIM_Provisioned" + +# Enterprise role capability edges +CREATE_ENTERPRISE_ORGANIZATIONS = "GH_CreateEnterpriseOrganizations" +EDIT_ENTERPRISE_CUSTOM_PROPERTIES_FOR_ORGANIZATIONS = ( + "GH_EditEnterpriseCustomPropertiesForOrganizations" +) +MANAGE_ENTERPRISE_ADMINS = "GH_ManageEnterpriseAdmins" +MANAGE_ENTERPRISE_IDENTITY_PROVIDER = "GH_ManageEnterpriseIdentityProvider" +MANAGE_ENTERPRISE_MEMBERS = "GH_ManageEnterpriseMembers" +MANAGE_ENTERPRISE_ORGANIZATION_ADMINS = "GH_ManageEnterpriseOrganizationAdmins" +MANAGE_ENTERPRISE_ORGANIZATIONS = "GH_ManageEnterpriseOrganizations" +MANAGE_ENTERPRISE_REFERRALS = "GH_ManageEnterpriseReferrals" +MANAGE_ENTERPRISE_TEAMS = "GH_ManageEnterpriseTeams" +READ_ENTERPRISE_AUDIT_LOG = "GH_ReadEnterpriseAuditLog" +READ_ENTERPRISE_DOMAIN_VERIFICATION = "GH_ReadEnterpriseDomainVerification" +READ_ENTERPRISE_MEMBERS = "GH_ReadEnterpriseMembers" +READ_ENTERPRISE_ORG_PROJECTS = "GH_ReadEnterpriseOrgProjects" +READ_ENTERPRISE_ORGANIZATION_ADMIN = "GH_ReadEnterpriseOrganizationAdmin" +SET_ENTERPRISE_INTERACTION_LIMITS = "GH_SetEnterpriseInteractionLimits" +VIEW_ENTERPRISE_ACTIONS_USAGE_METRICS = "GH_ViewEnterpriseActionsUsageMetrics" +VIEW_ENTERPRISE_BILLING = "GH_ViewEnterpriseBilling" +VIEW_ENTERPRISE_SECRET_SCANNING_ALERTS = "GH_ViewEnterpriseSecretScanningAlerts" +WRITE_ENTERPRISE_ACTIONS_POLICIES = "GH_WriteEnterpriseActionsPolicies" +WRITE_ENTERPRISE_BILLING = "GH_WriteEnterpriseBilling" +WRITE_ENTERPRISE_PERSONAL_ACCESS_TOKEN_POLICIES = ( + "GH_WriteEnterprisePersonalAccessTokenPolicies" +) +WRITE_ENTERPRISE_SSO = "GH_WriteEnterpriseSso" +WRITE_ENTERPRISE_TEAM_MEMBERS = "GH_WriteEnterpriseTeamMembers" + # Personal access token edges HAS_PERSONAL_ACCESS_TOKEN = "GH_HasPersonalAccessToken" HAS_PERSONAL_ACCESS_TOKEN_REQUEST = "GH_HasPersonalAccessTokenRequest" @@ -49,15 +87,15 @@ # Other HAS_SECRET_SCANNING_ALERT = "GH_HasSecretScanningAlert" VALID_TOKEN = "GH_ValidToken" -HAS_WORKFLOW = "GH_HasWorkflow" -HAS_JOB = "GH_HasJob" -HAS_STEP = "GH_HasStep" DEPENDS_ON = "GH_DependsOn" DEPLOYS_TO = "GH_DeploysTo" +APPROVES_DEPLOYMENT_TO = "GH_ApprovesDeploymentTo" CALLS_WORKFLOW = "GH_CallsWorkflow" USES_SECRET = "GH_UsesSecret" USES_VARIABLE = "GH_UsesVariable" HAS_ENVIRONMENT = "GH_HasEnvironment" +CAN_DEPLOY_TO_ENVIRONMENT = "GH_CanDeployToEnvironment" +MATCHES_ENVIRONMENT_POLICY = "GH_MatchesEnvironmentPolicy" INVITE_MEMBER = "GH_InviteMember" @@ -143,3 +181,15 @@ CAN_PWN_REQUEST = "GH_CanPwnRequest" CAN_READ_SECRET = "GH_CanReadSecret" CAN_READ_SECRET_SCANNING_ALERT = "GH_CanReadSecretScanningAlert" + +# SAML edges +SAML_IMPLEMENTS = "SAML_Implements" +SAML_ISSUES_AS = "SAML_IssuesAs" +SAML_TRUSTS_ISSUER = "SAML_TrustsIssuer" +SAML_ISSUES_ASSERTIONS_TO = "SAML_IssuesAssertionsTo" +SAML_HAS_ASSERTION_CONSUMER_SERVICE = "SAML_HasAssertionConsumerService" +SAML_HAS_CLAIM_MAPPING = "SAML_HasClaimMapping" +SAML_HAS_ACCOUNT = "SAML_HasAccount" +SAML_ELIGIBLE_FOR = "SAML_EligibleFor" +SAML_ISSUES_ASSERTIONS_FOR = "SAML_IssuesAssertionsFor" +SAML_CAN_SIGN_IN_AS = "SAML_CanSignInAs" diff --git a/src/openhound_github/kinds/nodes.py b/src/openhound_github/kinds/nodes.py index ce75797..e1e4443 100644 --- a/src/openhound_github/kinds/nodes.py +++ b/src/openhound_github/kinds/nodes.py @@ -10,6 +10,7 @@ ENVIRONMENT = "GH_Environment" ENVIRONMENT_SECRET = "GH_EnvironmentSecret" ENVIRONMENT_VARIABLE = "GH_EnvironmentVariable" +ENVIRONMENT_BRANCH_POLICY = "GH_EnvironmentBranchPolicy" # Identity nodes EXTERNAL_IDENTITY = "GH_ExternalIdentity" @@ -58,8 +59,25 @@ WORKFLOW_JOB = "GH_WorkflowJob" WORKFLOW_STEP = "GH_WorkflowStep" - # Enterprise nodes ENTERPRISE = "GH_Enterprise" ENTERPRISE_TEAM = "GH_EnterpriseTeam" ENTERPRISE_ROLE = "GH_EnterpriseRole" + +# SAML nodes +SAML_FEDERATION_PROVIDER = "SAML_FederationProvider" +SAML_SERVICE_PROVIDER = "SAML_ServiceProvider" +SAML_ISSUER = "SAML_Issuer" +SAML_ASSERTION_CONSUMER_SERVICE = "SAML_AssertionConsumerService" +SAML_CLAIM_MAPPING = "SAML_ClaimMapping" + +# Normalized OpenGraph-SCIM nodes +SCIM_ORGANIZATION = "SCIM_Organization" +SCIM_USER = "SCIM_User" +SCIM_GROUP = "SCIM_Group" +SCIM_ROLE = "SCIM_Role" + +# External node kinds used only as correlation endpoints +AZ_USER = "AZUser" +OKTA_USER = "Okta_User" +PINGONE_USER = "PingOne_User" diff --git a/src/openhound_github/lookup.py b/src/openhound_github/lookup.py index cc296dc..3a8b334 100644 --- a/src/openhound_github/lookup.py +++ b/src/openhound_github/lookup.py @@ -51,6 +51,30 @@ def enterprise_id(self) -> str | None: res = self._find_single_object(f"""SELECT id FROM {self.schema}.enterprise""") return res + @lru_cache + def enterprise_idp_for_scope( + self, enterprise_node_id: str + ) -> tuple[str | None, str | None] | None: + return self._find_single_row( + f"""SELECT issuer, sso_url + FROM {self.schema}.saml_provider + WHERE environment_node_id = ? AND environment_type = 'enterprise' + LIMIT 1""", + [enterprise_node_id], + ) + + @lru_cache + def warn_missing_legacy_scim_okta_tenant_once( + self, enterprise_node_id: str, enterprise_name: str + ) -> None: + logger.warning( + "Legacy SCIM correlations are enabled for GitHub enterprise '%s' " + "(%s), but no Okta tenant could be derived from its SAML provider; " + "skipping IdP-to-SCIM group edges.", + enterprise_name, + enterprise_node_id, + ) + @lru_cache def org_login_for_id(self, org_node_id: str) -> str | None: return self._find_single_object( @@ -98,10 +122,10 @@ def idp(self) -> list: ) @lru_cache - def idp_for_org(self, org_login: str) -> list: + def idp_for_environment(self, environment_slug: str) -> list: return self._find_all_objects( - f"""SELECT id, issuer, sso_url FROM {self.schema}.saml_provider WHERE org_login = ?""", - [org_login], + f"""SELECT id, issuer, sso_url, environment_node_id, environment_name, environment_type FROM {self.schema}.saml_provider WHERE environment_slug = ?""", + [environment_slug], ) @lru_cache @@ -394,6 +418,19 @@ def environment(self, env_name: str, repository_id: str): [env_name, repository_id], ) + @lru_cache + def environment_deployment_branch_policy( + self, environment_name: str, repository_node_id: str + ) -> tuple[bool, bool] | None: + return self._find_single_row( + f"""SELECT + coalesce(deployment_branch_policy->>'protected_branches', 'false') = 'true' AS protected_branches, + coalesce(deployment_branch_policy->>'custom_branch_policies', 'false') = 'true' AS custom_branch_policies + FROM {self.schema}.environments + WHERE name = ? AND repository_node_id = ?""", + [environment_name, repository_node_id], + ) + @lru_cache def workflow(self, repository_node_id: str, path: str): return self._find_single_object( @@ -407,7 +444,12 @@ def workflow(self, repository_node_id: str, path: str): @lru_cache def branches_for_repository(self, repository_node_id: str): return self._find_all_objects( - f"""SELECT id, name FROM {self.schema}.branches WHERE repository_node_id = ?""", + f"""SELECT + id, + name, + branch_protection_rule IS NOT NULL AS protected + FROM {self.schema}.branches + WHERE repository_node_id = ?""", [repository_node_id], ) diff --git a/src/openhound_github/models/__init__.py b/src/openhound_github/models/__init__.py index 1476233..2002c90 100644 --- a/src/openhound_github/models/__init__.py +++ b/src/openhound_github/models/__init__.py @@ -6,7 +6,6 @@ from .branch_push_allowance import BranchPushAllowance from .enterprise import Enterprise from .enterprise_admin import EnterpriseAdmin -from .enterprise_external_identity import EnterpriseExternalIdentity from .enterprise_helpers import enterprise_role_node_id, enterprise_team_node_id from .enterprise_managed_user import EnterpriseManagedUser from .enterprise_member import BaseUser, flatten_enterprise_member @@ -14,7 +13,6 @@ from .enterprise_role import EnterpriseRole from .enterprise_role_team import EnterpriseRoleTeam from .enterprise_role_user import EnterpriseRoleUser -from .enterprise_saml_provider import EnterpriseSamlProvider from .enterprise_team import EnterpriseTeam from .enterprise_team_member import EnterpriseTeamMember from .enterprise_team_organization import EnterpriseTeamOrganization @@ -41,8 +39,11 @@ from .repository_secret import RepoSecret from .repository_variable import RepoVariable from .runner import OrgRunner, OrgRunnerGroupMembership, RepoRunner, RunnerGroup +from .saml_assertion_consumer_service import SamlAssertionConsumerService from .saml_provider import SamlProvider -from .scim_user import ScimResource +from .saml_service_provider import SamlServiceProvider +from .saml_issuer import SamlIssuer +from .scim_user import ScimGroup, ScimOrganization, ScimResource, ScimUser from .secret_scanning_alert import SecretScanningAlert from .team import Team from .team_member import TeamMember @@ -92,9 +93,15 @@ "ProjectedEnterpriseTeam", "SelectedOrgSecret", "PersonalAccessTokenRequest", + "SamlAssertionConsumerService", "SamlProvider", + "SamlServiceProvider", + "SamlIssuer", "ExternalIdentity", "ScimResource", + "ScimUser", + "ScimGroup", + "ScimOrganization", "RepoRoleAssignment", "Environment", "EnvironmentSecret", @@ -102,13 +109,11 @@ "EnvironmentBranchPolicy", "Enterprise", "EnterpriseAdmin", - "EnterpriseExternalIdentity", "EnterpriseManagedUser", "EnterpriseOrganization", "EnterpriseRole", "EnterpriseRoleTeam", "EnterpriseRoleUser", - "EnterpriseSamlProvider", "EnterpriseTeam", "EnterpriseTeamMember", "EnterpriseTeamOrganization", diff --git a/src/openhound_github/models/branch.py b/src/openhound_github/models/branch.py index 4de85cb..3cc20ae 100644 --- a/src/openhound_github/models/branch.py +++ b/src/openhound_github/models/branch.py @@ -50,8 +50,8 @@ class ProtectionRule(BaseModel): EdgeDef( start=nk.REPOSITORY, end=nk.BRANCH, - kind=ek.HAS_BRANCH, - description="Repository has branch", + kind=ek.CONTAINS, + description="Repository contains branch", traversable=False, ), EdgeDef( @@ -59,12 +59,12 @@ class ProtectionRule(BaseModel): end=nk.BRANCH, kind=ek.PROTECTED_BY, description="Branch is protected by rule", - traversable=True, + traversable=False, ), ], ) class Branch(BaseAsset): - """One record from the `branches` DLT table → one GH_Branch node + GH_HasBranch edge + optional GH_ProtectedBy edge.""" + """One record from the `branches` DLT table → one GH_Branch node + GH_Contains edge + optional GH_ProtectedBy edge.""" model_config = ConfigDict(populate_by_name=True) id: str @@ -123,7 +123,7 @@ def _protection_edge(self): @property def edges(self): yield Edge( - kind=ek.HAS_BRANCH, + kind=ek.CONTAINS, start=EdgePath(value=self.repository_node_id, match_by="id"), end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), diff --git a/src/openhound_github/models/enterprise_external_identity.py b/src/openhound_github/models/enterprise_external_identity.py deleted file mode 100644 index 52120f2..0000000 --- a/src/openhound_github/models/enterprise_external_identity.py +++ /dev/null @@ -1,202 +0,0 @@ -from dataclasses import dataclass - -from openhound.core.asset import BaseAsset, EdgeDef, NodeDef -from openhound.core.models.entries_dataclass import ( - ConditionalEdgePath, - Edge, - EdgePath, - EdgeProperties, - PropertyMatch, -) -from pydantic import BaseModel, ConfigDict, Field - -from openhound_github.graph import GHNode, GHNodeProperties -from openhound_github.kinds import edges as ek -from openhound_github.kinds import nodes as nk -from openhound_github.main import app -from openhound_github.models.enterprise_saml_provider import EnterpriseSamlProvider - - -class EnterpriseIdentityName(BaseModel): - model_config = ConfigDict(extra="allow", populate_by_name=True) - - family_name: str | None = Field(alias="familyName", default=None) - given_name: str | None = Field(alias="givenName", default=None) - name_id: str | None = Field(alias="nameId", default=None) - username: str | None = None - - -class EnterpriseIdentityUser(BaseModel): - id: str - login: str - - -@dataclass -class GHEnterpriseExternalIdentityProperties(GHNodeProperties): - """Properties for an enterprise external identity. - - Attributes: - guid: The external identity GUID. - saml_identity_username: The SAML username. - saml_identity_name_id: The SAML NameID. - saml_identity_given_name: The SAML given name. - saml_identity_family_name: The SAML family name. - scim_identity_username: The SCIM username. - scim_identity_given_name: The SCIM given name. - scim_identity_family_name: The SCIM family name. - github_username: The mapped GitHub username. - github_user_id: The mapped GitHub user node ID. - environment_name: The enterprise environment name. - query_mapped_users: Query for mapped users. - """ - - guid: str | None = None - saml_identity_username: str | None = None - saml_identity_name_id: str | None = None - saml_identity_given_name: str | None = None - saml_identity_family_name: str | None = None - scim_identity_username: str | None = None - scim_identity_given_name: str | None = None - scim_identity_family_name: str | None = None - github_username: str | None = None - github_user_id: str | None = None - environment_name: str | None = None - query_mapped_users: str | None = None - - -@app.asset( - node=NodeDef( - kind=nk.EXTERNAL_IDENTITY, - description="GitHub Enterprise External Identity", - icon="arrows-left-right", - properties=GHEnterpriseExternalIdentityProperties, - ), - edges=[ - EdgeDef( - start=nk.SAML_IDENTITY_PROVIDER, - end=nk.EXTERNAL_IDENTITY, - kind=ek.HAS_EXTERNAL_IDENTITY, - description="Enterprise IdP has external identity", - traversable=False, - ), - EdgeDef( - start=nk.EXTERNAL_IDENTITY, - end=nk.USER, - kind=ek.MAPS_TO_USER, - description="External identity maps to GitHub user", - traversable=False, - ), - ], -) -class EnterpriseExternalIdentity(BaseAsset): - model_config = ConfigDict(extra="allow", populate_by_name=True) - - guid: str | None = None - id: str - saml_identity: EnterpriseIdentityName | None = Field( - alias="samlIdentity", default=None - ) - scim_identity: EnterpriseIdentityName | None = Field( - alias="scimIdentity", default=None - ) - user: EnterpriseIdentityUser | None = None - saml_provider_id: str - saml_provider_issuer: str | None = None - saml_provider_sso_url: str | None = None - enterprise_node_id: str - enterprise_slug: str - - @property - def node_id(self) -> str: - return self.id - - @property - def foreign_user(self) -> tuple[str | None, str | None]: - return EnterpriseSamlProvider.detect_foreign_environment( - self.saml_provider_issuer, self.saml_provider_sso_url - ) - - @property - def foreign_username(self) -> str | None: - if self.saml_identity and self.saml_identity.username: - return self.saml_identity.username - if self.scim_identity and self.scim_identity.username: - return self.scim_identity.username - return None - - @property - def as_node(self) -> GHNode: - return GHNode( - kinds=[nk.EXTERNAL_IDENTITY], - properties=GHEnterpriseExternalIdentityProperties( - name=self.guid or self.node_id, - displayname=self.guid or self.node_id, - node_id=self.node_id, - environmentid=self._lookup.enterprise_id(), - environment_name=self.enterprise_slug, - guid=self.guid, - saml_identity_username=self.saml_identity.username - if self.saml_identity - else None, - saml_identity_name_id=self.saml_identity.name_id - if self.saml_identity - else None, - saml_identity_given_name=self.saml_identity.given_name - if self.saml_identity - else None, - saml_identity_family_name=self.saml_identity.family_name - if self.saml_identity - else None, - scim_identity_username=self.scim_identity.username - if self.scim_identity - else None, - scim_identity_given_name=self.scim_identity.given_name - if self.scim_identity - else None, - scim_identity_family_name=self.scim_identity.family_name - if self.scim_identity - else None, - github_username=self.user.login if self.user else None, - github_user_id=self.user.id if self.user else None, - query_mapped_users=f"MATCH p=(:GH_ExternalIdentity {{node_id:'{self.node_id}'}})-[:GH_MapsToUser]->() RETURN p", - ), - ) - - @property - def edges(self): - yield Edge( - kind=ek.HAS_EXTERNAL_IDENTITY, - start=EdgePath(value=self.saml_provider_id, match_by="id"), - end=EdgePath(value=self.node_id, match_by="id"), - properties=EdgeProperties(traversable=False), - ) - if self.user and self.user.id: - yield Edge( - kind=ek.MAPS_TO_USER, - start=EdgePath(value=self.node_id, match_by="id"), - end=EdgePath(value=self.user.id, match_by="id"), - properties=EdgeProperties(traversable=False), - ) - - foreign_kind, _ = self.foreign_user - if foreign_kind and self.foreign_username: - match = PropertyMatch(key="name", value=self.foreign_username.upper()) - yield Edge( - kind=ek.MAPS_TO_USER, - start=EdgePath(value=self.node_id, match_by="id"), - end=ConditionalEdgePath( - kind=foreign_kind, - property_matchers=[match], - ), - properties=EdgeProperties(traversable=False), - ) - if self.user and self.user.id: - yield Edge( - kind=ek.SYNCED_TO_GH_USER, - start=ConditionalEdgePath( - kind=foreign_kind, - property_matchers=[match], - ), - end=EdgePath(value=self.user.id, match_by="id"), - properties=EdgeProperties(traversable=True), - ) diff --git a/src/openhound_github/models/enterprise_role.py b/src/openhound_github/models/enterprise_role.py index 5700056..2fae124 100644 --- a/src/openhound_github/models/enterprise_role.py +++ b/src/openhound_github/models/enterprise_role.py @@ -13,6 +13,63 @@ from openhound_github.models.enterprise_helpers import enterprise_role_node_id +ENTERPRISE_PERMISSION_EDGES: dict[str, tuple[str, bool]] = { + "create_enterprise_organizations": (ek.CREATE_ENTERPRISE_ORGANIZATIONS, False), + "edit_enterprise_custom_properties_for_organizations": ( + ek.EDIT_ENTERPRISE_CUSTOM_PROPERTIES_FOR_ORGANIZATIONS, + False, + ), + "manage_enterprise_admins": (ek.MANAGE_ENTERPRISE_ADMINS, True), + "manage_enterprise_identity_provider": ( + ek.MANAGE_ENTERPRISE_IDENTITY_PROVIDER, + False, + ), + "manage_enterprise_members": (ek.MANAGE_ENTERPRISE_MEMBERS, True), + "manage_enterprise_organization_admins": ( + ek.MANAGE_ENTERPRISE_ORGANIZATION_ADMINS, + True, + ), + "manage_enterprise_organizations": (ek.MANAGE_ENTERPRISE_ORGANIZATIONS, False), + "manage_enterprise_referrals": (ek.MANAGE_ENTERPRISE_REFERRALS, False), + "manage_enterprise_teams": (ek.MANAGE_ENTERPRISE_TEAMS, False), + "read_enterprise_audit_log": (ek.READ_ENTERPRISE_AUDIT_LOG, False), + "read_enterprise_domain_verification": ( + ek.READ_ENTERPRISE_DOMAIN_VERIFICATION, + False, + ), + "read_enterprise_members": (ek.READ_ENTERPRISE_MEMBERS, False), + "read_enterprise_org_projects": (ek.READ_ENTERPRISE_ORG_PROJECTS, False), + "read_enterprise_organization_admin": ( + ek.READ_ENTERPRISE_ORGANIZATION_ADMIN, + False, + ), + "set_enterprise_interaction_limits": ( + ek.SET_ENTERPRISE_INTERACTION_LIMITS, + False, + ), + "view_enterprise_actions_usage_metrics": ( + ek.VIEW_ENTERPRISE_ACTIONS_USAGE_METRICS, + False, + ), + "view_enterprise_billing": (ek.VIEW_ENTERPRISE_BILLING, False), + "view_enterprise_secret_scanning_alerts": ( + ek.VIEW_ENTERPRISE_SECRET_SCANNING_ALERTS, + False, + ), + "write_enterprise_actions_policies": ( + ek.WRITE_ENTERPRISE_ACTIONS_POLICIES, + False, + ), + "write_enterprise_billing": (ek.WRITE_ENTERPRISE_BILLING, False), + "write_enterprise_personal_access_token_policies": ( + ek.WRITE_ENTERPRISE_PERSONAL_ACCESS_TOKEN_POLICIES, + False, + ), + "write_enterprise_sso": (ek.WRITE_ENTERPRISE_SSO, False), + "write_enterprise_team_members": (ek.WRITE_ENTERPRISE_TEAM_MEMBERS, False), +} + + @dataclass class GHEnterpriseRoleProperties(GHNodeProperties): """Properties for a GitHub enterprise role. @@ -61,6 +118,16 @@ class GHEnterpriseRoleProperties(GHNodeProperties): description="Enterprise contains role", traversable=False, ), + *[ + EdgeDef( + start=nk.ENTERPRISE_ROLE, + end=nk.ENTERPRISE, + kind=edge_kind, + description=f"Enterprise role has {permission} capability", + traversable=traversable, + ) + for permission, (edge_kind, traversable) in ENTERPRISE_PERMISSION_EDGES.items() + ], ], ) class EnterpriseRole(BaseAsset): @@ -118,3 +185,14 @@ def edges(self): end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), ) + for permission in self.permissions: + edge_definition = ENTERPRISE_PERMISSION_EDGES.get(permission) + if not edge_definition: + continue + edge_kind, traversable = edge_definition + yield Edge( + kind=edge_kind, + start=EdgePath(value=self.node_id, match_by="id"), + end=EdgePath(value=self.enterprise_node_id, match_by="id"), + properties=EdgeProperties(traversable=traversable), + ) diff --git a/src/openhound_github/models/enterprise_saml_provider.py b/src/openhound_github/models/enterprise_saml_provider.py deleted file mode 100644 index 4a30ed0..0000000 --- a/src/openhound_github/models/enterprise_saml_provider.py +++ /dev/null @@ -1,122 +0,0 @@ -from dataclasses import dataclass -from typing import ClassVar - -from dlt.common.libs.pydantic import DltConfig -from openhound.core.asset import BaseAsset, EdgeDef, NodeDef -from openhound.core.models.entries_dataclass import Edge, EdgePath, EdgeProperties -from pydantic import ConfigDict, Field - -from openhound_github.graph import GHNode, GHNodeProperties -from openhound_github.kinds import edges as ek -from openhound_github.kinds import nodes as nk -from openhound_github.main import app - - -@dataclass -class GHEnterpriseSamlProviderProperties(GHNodeProperties): - """Properties for an enterprise SAML identity provider. - - Attributes: - issuer: The SAML issuer. - sso_url: The SAML SSO URL. - signature_method: The SAML signature method. - digest_method: The SAML digest method. - idp_certificate: The IdP certificate. - foreign_environment_id: The correlated foreign environment ID. - environment_name: The enterprise environment name. - query_environments: Query for owning environments. - query_external_identities: Query for external identities. - """ - - issuer: str | None = None - sso_url: str | None = None - signature_method: str | None = None - digest_method: str | None = None - idp_certificate: str | None = None - foreign_environment_id: str | None = None - environment_name: str | None = None - query_environments: str | None = None - query_external_identities: str | None = None - - -@app.asset( - node=NodeDef( - kind=nk.SAML_IDENTITY_PROVIDER, - description="GitHub Enterprise SAML Identity Provider", - icon="id-badge", - properties=GHEnterpriseSamlProviderProperties, - ), - edges=[ - EdgeDef( - start=nk.ENTERPRISE, - end=nk.SAML_IDENTITY_PROVIDER, - kind=ek.HAS_SAML_IDENTITY_PROVIDER, - description="Enterprise uses this SAML IdP", - traversable=False, - ), - ], -) -class EnterpriseSamlProvider(BaseAsset): - model_config = ConfigDict(extra="allow", populate_by_name=True) - - id: str - issuer: str | None = None - sso_url: str | None = Field(alias="ssoUrl", default=None) - digest_method: str | None = Field(alias="digestMethod", default=None) - signature_method: str | None = Field(alias="signatureMethod", default=None) - idp_certificate: str | None = Field(alias="idpCertificate", default=None) - enterprise_node_id: str - enterprise_slug: str - - dlt_config: ClassVar[DltConfig] = {"return_validated_models": True} - - @property - def node_id(self) -> str: - return self.id - - @staticmethod - def detect_foreign_environment( - issuer: str | None, sso_url: str | None - ) -> tuple[str | None, str | None]: - if not issuer: - return None, None - if issuer.startswith("https://auth.pingone.com/"): - return "PingOneUser", issuer.split("/")[3] - if issuer.startswith("https://sts.windows.net/"): - return "AZUser", issuer.split("/")[3] - if issuer.startswith("http://www.okta.com/"): - return "Okta_User", sso_url.split("/")[2] if sso_url else None - return None, None - - @property - def as_node(self) -> GHNode: - _, foreign_environment_id = self.detect_foreign_environment( - self.issuer, self.sso_url - ) - return GHNode( - kinds=[nk.SAML_IDENTITY_PROVIDER], - properties=GHEnterpriseSamlProviderProperties( - name=self.node_id, - displayname=self.enterprise_slug, - node_id=self.node_id, - environmentid=self._lookup.enterprise_id(), - environment_name=self.enterprise_slug, - issuer=self.issuer, - sso_url=self.sso_url, - signature_method=self.signature_method, - digest_method=self.digest_method, - idp_certificate=self.idp_certificate, - foreign_environment_id=foreign_environment_id, - query_environments=f"MATCH p=(:GH_SamlIdentityProvider {{node_id:'{self.node_id}'}})<-[:GH_HasSamlIdentityProvider]-(:GH_Enterprise) RETURN p", - query_external_identities=f"MATCH p=(:GH_SamlIdentityProvider {{node_id:'{self.node_id}'}})-[:GH_HasExternalIdentity]->() RETURN p", - ), - ) - - @property - def edges(self): - yield Edge( - kind=ek.HAS_SAML_IDENTITY_PROVIDER, - start=EdgePath(value=self._lookup.enterprise_id(), match_by="id"), - end=EdgePath(value=self.node_id, match_by="id"), - properties=EdgeProperties(traversable=False), - ) diff --git a/src/openhound_github/models/enterprise_team.py b/src/openhound_github/models/enterprise_team.py index 7b283a1..7f4e956 100644 --- a/src/openhound_github/models/enterprise_team.py +++ b/src/openhound_github/models/enterprise_team.py @@ -61,6 +61,13 @@ class GHEnterpriseTeamProperties(GHNodeProperties): description="Enterprise contains team", traversable=False, ), + EdgeDef( + start=nk.SCIM_GROUP, + end=nk.ENTERPRISE_TEAM, + kind=ek.SCIM_PROVISIONED, + description="SCIM group is provisioned as enterprise team", + traversable=True, + ), ], ) class EnterpriseTeam(BaseAsset): @@ -114,3 +121,10 @@ def edges(self): end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), ) + if self.group_id: + yield Edge( + kind=ek.SCIM_PROVISIONED, + start=EdgePath(value=self.group_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) diff --git a/src/openhound_github/models/enterprise_user.py b/src/openhound_github/models/enterprise_user.py index 049db91..d588c50 100644 --- a/src/openhound_github/models/enterprise_user.py +++ b/src/openhound_github/models/enterprise_user.py @@ -8,6 +8,7 @@ from openhound_github.kinds import edges as ek from openhound_github.kinds import nodes as nk from openhound_github.main import app +from openhound_github.models.enterprise_helpers import enterprise_role_node_id @dataclass @@ -48,6 +49,13 @@ class GHEnterpriseUserProperties(GHNodeProperties): description="Enterprise has user member", traversable=False, ), + EdgeDef( + start=nk.USER, + end=nk.ENTERPRISE_ROLE, + kind=ek.HAS_ROLE, + description="Enterprise user has built-in members role", + traversable=True, + ), ], ) class EnterpriseUser(BaseAsset): @@ -94,3 +102,14 @@ def edges(self): end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), ) + yield Edge( + kind=ek.HAS_ROLE, + start=EdgePath(value=self.node_id, match_by="id"), + end=EdgePath( + value=enterprise_role_node_id( + self._lookup.enterprise_id(), "members" + ), + match_by="id", + ), + properties=EdgeProperties(traversable=True), + ) diff --git a/src/openhound_github/models/env_secret.py b/src/openhound_github/models/env_secret.py index 3945db1..819cdc8 100644 --- a/src/openhound_github/models/env_secret.py +++ b/src/openhound_github/models/env_secret.py @@ -47,7 +47,14 @@ class GHEnvironmentSecretProperties(GHNodeProperties): kind=ek.CONTAINS, description="Environment contains secret", traversable=False, - ) + ), + EdgeDef( + start=nk.ENVIRONMENT, + end=nk.ENVIRONMENT_SECRET, + kind=ek.HAS_SECRET, + description="Environment has secret", + traversable=True, + ), ], ) class EnvironmentSecret(BaseAsset): @@ -102,3 +109,9 @@ def edges(self): end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), ) + yield Edge( + kind=ek.HAS_SECRET, + start=EdgePath(value=self.environment_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) diff --git a/src/openhound_github/models/env_variable.py b/src/openhound_github/models/env_variable.py index 296955f..06b3b0e 100644 --- a/src/openhound_github/models/env_variable.py +++ b/src/openhound_github/models/env_variable.py @@ -51,7 +51,14 @@ class GHEnvVariableProperties(GHNodeProperties): kind=ek.CONTAINS, description="Environment contains variable", traversable=False, - ) + ), + EdgeDef( + start=nk.ENVIRONMENT, + end=nk.ENVIRONMENT_VARIABLE, + kind=ek.HAS_VARIABLE, + description="Environment has variable", + traversable=True, + ), ], ) class EnvironmentVariable(BaseAsset): @@ -106,3 +113,9 @@ def edges(self): end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), ) + yield Edge( + kind=ek.HAS_VARIABLE, + start=EdgePath(value=self.environment_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) diff --git a/src/openhound_github/models/environment.py b/src/openhound_github/models/environment.py index 7191be1..174677d 100644 --- a/src/openhound_github/models/environment.py +++ b/src/openhound_github/models/environment.py @@ -6,7 +6,7 @@ from openhound.core.models.entries_dataclass import Edge, EdgePath, EdgeProperties from pydantic import BaseModel -from openhound_github.graph import GHNode, GHNodeProperties +from openhound_github.graph import GHEdgeProperties, GHNode, GHNodeProperties from openhound_github.kinds import edges as ek from openhound_github.kinds import nodes as nk from openhound_github.main import app @@ -71,7 +71,7 @@ class ProtectionRule(BaseModel): @dataclass class GHEnvironmentProperties(GHNodeProperties): """Environment-specific properties and accordion panel queries. - + Attributes: collected: Collected/generated by OpenHound short_name: The environment's display name (e.g., `production`, `staging`). @@ -84,6 +84,12 @@ class GHEnvironmentProperties(GHNodeProperties): collected: bool = True short_name: str | None = None can_admins_bypass: bool | None = None + wait_timer: int | None = None + required_reviewers: bool = False + reviewer_count: int = 0 + prevent_self_review: bool | None = None + protected_branches: bool | None = None + custom_branch_policies: bool | None = None repository_name: str | None = None repository_id: str | None = None environment_name: str | None = None @@ -100,17 +106,73 @@ class GHEnvironmentProperties(GHNodeProperties): EdgeDef( start=nk.REPOSITORY, end=nk.ENVIRONMENT, - kind=ek.HAS_ENVIRONMENT, - description="Repository deploys to environment (no custom branch policy)", + kind=ek.CONTAINS, + description="Repository contains environment", + traversable=False, + ), + EdgeDef( + start=nk.REPO_ROLE, + end=nk.ENVIRONMENT, + kind=ek.CAN_EDIT_ENVIRONMENT, + description="Repository admin role can edit environment configuration", + traversable=True, + ), + EdgeDef( + start=nk.REPOSITORY, + end=nk.ENVIRONMENT, + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + description="Repository can deploy to environment under current branch policy", + traversable=True, + ), + EdgeDef( + start=nk.BRANCH, + end=nk.ENVIRONMENT, + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + description="Branch can deploy to environment under current branch policy", + traversable=True, + ), + EdgeDef( + start=nk.REPO_ROLE, + end=nk.ENVIRONMENT, + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + description="Repository admin-equivalent role can bypass environment protections", traversable=True, ), + EdgeDef( + start=nk.USER, + end=nk.ENVIRONMENT, + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + description="Reviewer user can approve deployment to environment", + traversable=True, + ), + EdgeDef( + start=nk.TEAM, + end=nk.ENVIRONMENT, + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + description="Reviewer team can approve deployment to environment", + traversable=True, + ), + EdgeDef( + start=nk.USER, + end=nk.ENVIRONMENT, + kind=ek.APPROVES_DEPLOYMENT_TO, + description="User is configured as a required reviewer for this environment", + traversable=False, + ), + EdgeDef( + start=nk.TEAM, + end=nk.ENVIRONMENT, + kind=ek.APPROVES_DEPLOYMENT_TO, + description="Team is configured as a required reviewer for this environment", + traversable=False, + ), ], ) class Environment(BaseAsset): """One record from `environments` → one GH_Environment node. - GH_HasEnvironment edge from repo is only emitted when has_custom_branch_policies=False. - When True, EnvironmentBranchPolicyAsset emits the edges instead. + The repository always contains the environment. Deployment eligibility is + represented separately by computed GH_CanDeployToEnvironment edges. """ dlt_config: ClassVar[DltConfig] = {"return_validated_models": True} @@ -124,6 +186,7 @@ class Environment(BaseAsset): updated_at: str protection_rules: list[ProtectionRule] | None = None deployment_branch_policy: DeploymentBranchPolicy | None = None + can_admins_bypass: bool | None = None # Custom fields added org_login: str @@ -141,6 +204,34 @@ def has_custom_branch_policies(self) -> bool: return self.deployment_branch_policy.custom_branch_policies return False + @property + def wait_timer(self) -> int: + for rule in self.protection_rules or []: + if rule.type == "wait_timer": + return rule.wait_timer or 0 + return 0 + + @property + def required_reviewers(self) -> bool: + return any( + rule.type == "required_reviewers" + for rule in (self.protection_rules or []) + ) + + @property + def prevent_self_review(self) -> bool: + for rule in self.protection_rules or []: + if rule.type == "required_reviewers": + return bool(rule.prevent_self_review) + return False + + @property + def reviewers(self) -> list[Reviewer]: + for rule in self.protection_rules or []: + if rule.type == "required_reviewers": + return rule.reviewers or [] + return [] + @property def as_node(self) -> GHNode: eid = self.node_id @@ -151,7 +242,17 @@ def as_node(self) -> GHNode: displayname=self.name, node_id=eid, short_name=self.name, - # can_admins_bypass=self.can_admins_bypass, + can_admins_bypass=self.can_admins_bypass, + wait_timer=self.wait_timer, + required_reviewers=self.required_reviewers, + reviewer_count=len(self.reviewers), + prevent_self_review=self.prevent_self_review, + protected_branches=bool( + self.deployment_branch_policy.protected_branches + ) if self.deployment_branch_policy else False, + custom_branch_policies=bool( + self.deployment_branch_policy.custom_branch_policies + ) if self.deployment_branch_policy else False, repository_name=self.repository_name, repository_id=self.repository_node_id, environment_name=self.org_login, @@ -159,13 +260,195 @@ def as_node(self) -> GHNode: ), ) + def _repo_can_deploy_query(self) -> str: + return ( + f"MATCH p=(:GH_Repository {{node_id:'{self.repository_node_id}'}})" + f"-[:GH_Contains]->(env:GH_Environment {{node_id:'{self.node_id}'}}) " + f"WHERE coalesce(env.custom_branch_policies, false) = false " + f"AND coalesce(env.protected_branches, false) = false " + f"RETURN p" + ) + + def _branch_can_deploy_via_repo_query(self, branch_id: str) -> str: + return ( + f"MATCH p=(repo:GH_Repository {{node_id:'{self.repository_node_id}'}})" + f"-[:GH_Contains]->(:GH_Branch {{node_id:'{branch_id}'}}) " + f"MATCH p1=(repo)-[:GH_Contains]->" + f"(env:GH_Environment {{node_id:'{self.node_id}'}}) " + f"WHERE coalesce(env.custom_branch_policies, false) = false " + f"AND coalesce(env.protected_branches, false) = false " + f"RETURN p, p1" + ) + + def _protected_branch_can_deploy_query(self, branch_id: str) -> str: + return ( + f"MATCH p=(repo:GH_Repository {{node_id:'{self.repository_node_id}'}})" + f"-[:GH_Contains]->(:GH_Branch {{node_id:'{branch_id}'}})" + f"<-[:GH_ProtectedBy]-(:GH_BranchProtectionRule) " + f"MATCH p1=(repo)-[:GH_Contains]->" + f"(env:GH_Environment {{node_id:'{self.node_id}'}}) " + f"WHERE coalesce(env.custom_branch_policies, false) = false " + f"AND env.protected_branches = true " + f"RETURN p, p1" + ) + + def _admin_bypass_query(self) -> str: + admin_role_id = f"{self.repository_node_id}_admin" + return ( + f"MATCH p=(:GH_RepoRole {{node_id:'{admin_role_id}'}})" + f"-[:GH_AdminTo]->(:GH_Repository {{node_id:'{self.repository_node_id}'}})" + f"-[:GH_Contains]->(env:GH_Environment {{node_id:'{self.node_id}'}}) " + f"WHERE env.can_admins_bypass = true " + f"RETURN p" + ) + + def _protected_branches_fallback_repo_query(self) -> str: + return ( + f"MATCH p=(repo:GH_Repository {{node_id:'{self.repository_node_id}'}})" + f"-[:GH_Contains]->(env:GH_Environment {{node_id:'{self.node_id}'}}) " + f"WHERE env.protected_branches = true " + f"AND coalesce(env.custom_branch_policies, false) = false " + f"AND NOT EXISTS {{ " + f"MATCH (repo)-[:GH_Contains]->(:GH_Branch)" + f"<-[:GH_ProtectedBy]-(:GH_BranchProtectionRule) " + f"}} " + f"RETURN p" + ) + + def _protected_branches_fallback_branch_query(self, branch_id: str) -> str: + return ( + f"MATCH p=(repo:GH_Repository {{node_id:'{self.repository_node_id}'}})" + f"-[:GH_Contains]->(:GH_Branch {{node_id:'{branch_id}'}}) " + f"MATCH p1=(repo)-[:GH_Contains]->" + f"(env:GH_Environment {{node_id:'{self.node_id}'}}) " + f"WHERE env.protected_branches = true " + f"AND coalesce(env.custom_branch_policies, false) = false " + f"AND NOT EXISTS {{ " + f"MATCH (repo)-[:GH_Contains]->(:GH_Branch)" + f"<-[:GH_ProtectedBy]-(:GH_BranchProtectionRule) " + f"}} " + f"RETURN p, p1" + ) + @property def edges(self): - # TODO: Check if correct - if not self.has_custom_branch_policies: + yield Edge( + kind=ek.CONTAINS, + start=EdgePath(value=self.repository_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=EdgeProperties(traversable=False), + ) + + yield Edge( + kind=ek.CAN_EDIT_ENVIRONMENT, + start=EdgePath( + value=f"{self.repository_node_id}_admin", + match_by="id", + ), + end=EdgePath(value=self.node_id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) + + if not self.has_custom_branch_policies and not ( + self.deployment_branch_policy + and self.deployment_branch_policy.protected_branches + ): yield Edge( - kind=ek.HAS_ENVIRONMENT, + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, start=EdgePath(value=self.repository_node_id, match_by="id"), end=EdgePath(value=self.node_id, match_by="id"), - properties=EdgeProperties(traversable=True), + properties=GHEdgeProperties( + traversable=True, + composed=True, + query_composition=self._repo_can_deploy_query(), + ), + ) + + branches = self._lookup.branches_for_repository(self.repository_node_id) + for branch_id, _branch_name, _protected in branches: + yield Edge( + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + start=EdgePath(value=branch_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=GHEdgeProperties( + traversable=True, + composed=True, + query_composition=self._branch_can_deploy_via_repo_query(branch_id), + ), + ) + + if not self.has_custom_branch_policies and ( + self.deployment_branch_policy + and self.deployment_branch_policy.protected_branches + ): + protected_branches = self._lookup.branches_with_bpr(self.repository_node_id) + + if protected_branches: + for (branch_id,) in protected_branches: + yield Edge( + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + start=EdgePath(value=branch_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=GHEdgeProperties( + traversable=True, + composed=True, + query_composition=self._protected_branch_can_deploy_query(branch_id), + ), + ) + else: + yield Edge( + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + start=EdgePath(value=self.repository_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=GHEdgeProperties( + traversable=True, + composed=True, + query_composition=self._protected_branches_fallback_repo_query(), + ), + ) + + for branch_id, _branch_name, _protected in self._lookup.branches_for_repository( + self.repository_node_id + ): + yield Edge( + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + start=EdgePath(value=branch_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=GHEdgeProperties( + traversable=True, + composed=True, + query_composition=self._protected_branches_fallback_branch_query(branch_id), + ), + ) + + if self.can_admins_bypass: + yield Edge( + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + start=EdgePath( + value=f"{self.repository_node_id}_admin", + match_by="id", + ), + end=EdgePath(value=self.node_id, match_by="id"), + properties=GHEdgeProperties( + traversable=True, + composed=True, + query_composition=self._admin_bypass_query(), + ), + ) + + for reviewer in self.reviewers: + reviewer_data = reviewer.reviewer or {} + reviewer_node_id = reviewer_data.get("node_id") + if not reviewer_node_id: + continue + + reviewer_type = reviewer.type.lower() + if reviewer_type not in {"user", "team"}: + continue + + yield Edge( + kind=ek.APPROVES_DEPLOYMENT_TO, + start=EdgePath(value=reviewer_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=EdgeProperties(traversable=False), ) diff --git a/src/openhound_github/models/environment_branch_policy.py b/src/openhound_github/models/environment_branch_policy.py index 5a8bb00..92cd2b0 100644 --- a/src/openhound_github/models/environment_branch_policy.py +++ b/src/openhound_github/models/environment_branch_policy.py @@ -1,26 +1,47 @@ -from openhound.core.asset import BaseAsset, EdgeDef +from dataclasses import dataclass +from pathlib import PurePosixPath + +from openhound.core.asset import BaseAsset, EdgeDef, NodeDef from openhound.core.models.entries_dataclass import Edge, EdgePath, EdgeProperties +from openhound_github.graph import GHEdgeProperties, GHNode, GHNodeProperties from openhound_github.kinds import edges as ek from openhound_github.kinds import nodes as nk from openhound_github.main import app # from openhound_github.helpers import _b64 +@dataclass +class GHEnvironmentBranchPolicyProperties(GHNodeProperties): + environment_name: str | None = None + repository_name: str | None = None @app.asset( + node=NodeDef( + kind=nk.ENVIRONMENT_BRANCH_POLICY, + description="GitHub environment deployment branch policy", + icon="code-branch", + properties=GHEnvironmentBranchPolicyProperties, + ), edges=[ + EdgeDef( + start=nk.ENVIRONMENT, + end=nk.ENVIRONMENT_BRANCH_POLICY, + kind=ek.CONTAINS, + description="Environment contains deployment branch policy", + traversable=False, + ), EdgeDef( start=nk.BRANCH, - end=nk.ENVIRONMENT, - kind=ek.HAS_ENVIRONMENT, - description="Branch pattern can deploy to environment", + end=nk.ENVIRONMENT_BRANCH_POLICY, + kind=ek.MATCHES_ENVIRONMENT_POLICY, + description="Branch matches environment deployment policy", traversable=False, - ) + ), ], ) class EnvironmentBranchPolicy(BaseAsset): - """One record from `environment_branch_policies` → GH_HasEnvironment edge from synthetic policy ID. No node.""" + """One record from `environment_branch_policies` → GH_Contains edge from synthetic policy ID. No node.""" id: int node_id: str @@ -42,15 +63,90 @@ def policy_id(self) -> str: return f"{self.environment_node_id}_{self.name}" @property - def as_node(self) -> None: - return None + def environment_protected_branches(self) -> bool: + row = self._lookup.environment_deployment_branch_policy( + self.environment_name, self.repository_node_id + ) + if not row: + return False + protected_branches, _custom_branch_policies = row + return protected_branches + + def matches_branch(self, branch_name: str) -> bool: + return PurePosixPath(f"/{branch_name}").full_match( + f"/{self.name}", + case_sensitive=True, + ) + + @property + def as_node(self) -> GHNode: + return GHNode( + kinds=[nk.ENVIRONMENT_BRANCH_POLICY], + properties=GHEnvironmentBranchPolicyProperties( + name=self.name, + displayname=self.name, + node_id=self.node_id, + environment_name=self.environment_name, + repository_name=self.repository_name, + environmentid=self.environment_node_id, + ), + ) + + def _policy_branch_can_deploy_query(self, branch_id: str) -> str: + return ( + f"MATCH p=(:GH_Branch {{node_id:'{branch_id}'}})" + f"-[:GH_MatchesEnvironmentPolicy]->" + f"(:GH_EnvironmentBranchPolicy {{node_id:'{self.node_id}'}})" + f"<-[:GH_Contains]-(env:GH_Environment {{node_id:'{self.environment_node_id}'}}) " + f"WHERE env.custom_branch_policies = true " + f"AND coalesce(env.protected_branches, false) = false " + f"RETURN p" + ) + + def _protected_policy_branch_can_deploy_query(self, branch_id: str) -> str: + return ( + f"MATCH p=(:GH_Branch {{node_id:'{branch_id}'}})" + f"-[:GH_MatchesEnvironmentPolicy]->" + f"(:GH_EnvironmentBranchPolicy {{node_id:'{self.node_id}'}})" + f"<-[:GH_Contains]-(env:GH_Environment {{node_id:'{self.environment_node_id}'}}) " + f"MATCH p1=(repo:GH_Repository {{node_id:'{self.repository_node_id}'}})" + f"-[:GH_Contains]->(:GH_Branch {{node_id:'{branch_id}'}})" + f"<-[:GH_ProtectedBy]-(:GH_BranchProtectionRule) " + f"MATCH p2=(repo)-[:GH_Contains]->(env) " + f"WHERE env.custom_branch_policies = true " + f"AND env.protected_branches = true " + f"RETURN p, p1, p2" + ) @property def edges(self): - # policy_id = _b64(f"{self.environment_node_id}_{self.pattern}") yield Edge( - kind=ek.HAS_ENVIRONMENT, - start=EdgePath(value=self.policy_id, match_by="id"), - end=EdgePath(value=self.environment_node_id, match_by="id"), + kind=ek.CONTAINS, + start=EdgePath(value=self.environment_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), ) + branches = self._lookup.branches_for_repository(self.repository_node_id) + for branch_id, branch_name, protected in branches: + if self.matches_branch(branch_name): + yield Edge( + kind=ek.MATCHES_ENVIRONMENT_POLICY, + start=EdgePath(value=branch_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=EdgeProperties(traversable=False), + ) + if not self.environment_protected_branches or protected: + yield Edge( + kind=ek.CAN_DEPLOY_TO_ENVIRONMENT, + start=EdgePath(value=branch_id, match_by="id"), + end=EdgePath(value=self.environment_node_id, match_by="id"), + properties=GHEdgeProperties( + traversable=True, + composed=True, + query_composition=( + self._protected_policy_branch_can_deploy_query(branch_id) + if self.environment_protected_branches + else self._policy_branch_can_deploy_query(branch_id) + ), + ), + ) diff --git a/src/openhound_github/models/external_identity.py b/src/openhound_github/models/external_identity.py index 959eec1..9194d8a 100644 --- a/src/openhound_github/models/external_identity.py +++ b/src/openhound_github/models/external_identity.py @@ -1,4 +1,4 @@ -from dataclasses import dataclass +from dataclasses import dataclass, field from openhound.core.asset import BaseAsset, EdgeDef, NodeDef from openhound.core.models.entries_dataclass import ( @@ -6,7 +6,6 @@ Edge, EdgePath, EdgeProperties, - PropertyMatch, ) from pydantic import BaseModel, ConfigDict, Field @@ -15,11 +14,29 @@ from openhound_github.kinds import nodes as nk from openhound_github.main import app -_FOREIGN_USER_KIND: dict[str, str] = { - "entra": "AZUser", - "okta": "OktaUser", - "pingone": "PingOneUser", -} +from .saml_helpers import ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + ENTRA_OBJECT_ID_CLAIM, + SAML_CONTRACT_VERSION, + detect_foreign_idp, + foreign_user_kind, + foreign_user_matchers, + github_saml_service_provider_id, + saml_account_match_values, + saml_attribute_match_values, +) + + +@dataclass +class SAMLHasAccountEdgeProperties(EdgeProperties): + schema_contract_version: str = SAML_CONTRACT_VERSION + match_values: list[str] = field(default_factory=list) + scoped_exact_match_values: list[str] = field(default_factory=list) + entra_object_id_match_values: list[str] = field(default_factory=list) + direct_binding: bool = False + direct_binding_source: str | None = None + external_identity_id: str | None = None + account_state: str = "unknown" @dataclass @@ -37,7 +54,7 @@ class GHExternalIdentityProperties(GHNodeProperties): scim_identity_family_name: The family name from the SCIM identity. github_username: The GitHub login of the linked user. github_user_id: The GraphQL ID of the linked GitHub user. - environment_name: The name of the environment (GitHub organization). + environment_name: The name of the environment (GitHub organization or enterprise). query_mapped_users: Query for mapped users. """ @@ -56,16 +73,17 @@ class GHExternalIdentityProperties(GHNodeProperties): class SCIMIdentity(BaseModel): - family_name: str | None = Field(alias="FamilyName", default=None) + family_name: str | None = Field(alias="familyName", default=None) given_name: str | None = Field(alias="givenName", default=None) username: str | None = None class SAMLIdentity(BaseModel): - family_name: str | None = Field(alias="FamilyName", default=None) + family_name: str | None = Field(alias="familyName", default=None) given_name: str | None = Field(alias="givenName", default=None) name_id: str | None = Field(alias="nameId", default=None) username: str | None = None + attributes: list[dict[str, object]] = Field(default_factory=list) class User(BaseModel): @@ -102,6 +120,13 @@ class User(BaseModel): description="Foreign IdP user is synced to a GitHub user", traversable=True, ), + EdgeDef( + start=nk.SAML_SERVICE_PROVIDER, + end=nk.USER, + kind=ek.SAML_HAS_ACCOUNT, + description="Normalized SAML service provider has this downstream GitHub account", + traversable=False, + ), ], ) class ExternalIdentity(BaseAsset): @@ -111,16 +136,13 @@ class ExternalIdentity(BaseAsset): guid: str id: str - saml_identity: SAMLIdentity = Field(alias="samlIdentity") - scim_identity: SCIMIdentity | None = Field(alias="scimIdentity") + saml_identity: SAMLIdentity | None = Field(alias="samlIdentity", default=None) + scim_identity: SCIMIdentity | None = Field(alias="scimIdentity", default=None) user: User | None = None # Additional - org_login: str - - @property - def org_node_id(self) -> str | None: - return self._lookup.org_id_for_login(self.org_login) + environment_slug: str + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID @property def node_id(self) -> str: @@ -159,87 +181,148 @@ def as_node(self) -> GHNode: else None, github_username=self.user.login if self.user else None, github_user_id=self.user.id if self.user else None, - environment_name=self.org_login, - environmentid=self.org_node_id, - query_mapped_users=f"MATCH p=(:GH_ExternalIdentity {{node_id:'{self.node_id.upper()}'}})-[:GH_MapsToUser]->() RETURN p", + environment_name=self.idp["environment_name"] if self.idp else None, + environmentid=self.idp["environment_node_id"] if self.idp else None, + query_mapped_users=f"MATCH p=(:GH_ExternalIdentity {{node_id:'{self.node_id}'}})-[:GH_MapsToUser]->() RETURN p", ), ) - @staticmethod - def detect_foreign_idp( - issuer: str | None, sso_url: str | None - ) -> tuple[str | None, str | None]: - """Detect the foreign IdP type and tenant/environment ID from the issuer or SSO URL.""" - if not issuer: - return None, None - if issuer.startswith("https://auth.pingone.com/"): - return "pingone", issuer.split("/")[3] - if issuer.startswith("https://sts.windows.net/"): - return "entra", issuer.split("/")[3] - if issuer.startswith("http://www.okta.com/"): - domain = (sso_url or "").split("/")[2] if sso_url else None - return "okta", domain - return None, None - @property def idp(self) -> dict: - ext_idp = self._lookup.idp_for_org(self.org_login) + ext_idp = self._lookup.idp_for_environment(self.environment_slug) if not ext_idp: - return {"id": None, "issuer": None, "sso_url": None} - id, issuer, sso_url = ext_idp[0] + return { + "id": None, + "issuer": None, + "sso_url": None, + "environment_node_id": None, + "environment_name": None, + } + provider_id, issuer, sso_url, environment_node_id, environment_name, environment_type = ext_idp[0] return { - "id": id, + "id": provider_id, "issuer": issuer, "sso_url": sso_url, + "environment_node_id": environment_node_id, + "environment_name": environment_name, + "environment_type": environment_type, } @property def _maps_to_user_edges(self): - if self.saml_identity: - foreign_idp_type, foreign_env_id = self.detect_foreign_idp( - issuer=self.idp["issuer"], - sso_url=self.idp["sso_url"], + foreign_idp_type, foreign_env_id = detect_foreign_idp( + issuer=self.idp["issuer"], + sso_url=self.idp["sso_url"], + ) + foreign_kind = foreign_user_kind(foreign_idp_type) + + foreign_username = None + if self.saml_identity and self.saml_identity.username: + foreign_username = self.saml_identity.username + elif self.scim_identity and self.scim_identity.username: + foreign_username = self.scim_identity.username + + matchers = foreign_user_matchers( + foreign_kind, + foreign_env_id, + foreign_username, + self.saml_identity.attributes if self.saml_identity else [], + ) + if foreign_kind and matchers: + yield Edge( + kind=ek.MAPS_TO_USER, + start=EdgePath(value=self.node_id, match_by="id"), + end=ConditionalEdgePath( + kind=foreign_kind, + property_matchers=matchers, + ), + properties=EdgeProperties(traversable=False), + ) + + # SyncedToGHUser: foreign IdP user → GitHub user (traversable, with composition) + if matchers and self.user and self.user.id: + composition_matchers = [ + matcher for matcher in matchers if matcher.key in {"name", "objectid"} + ] + composition_predicates = " OR ".join( + f"n.{matcher.key} = '{matcher.value}'" + for matcher in composition_matchers ) - foreign_kind = _FOREIGN_USER_KIND.get(foreign_idp_type or "", "") - foreign_username = ( - self.saml_identity.username or self.scim_identity.username + q = ( + f"MATCH p=()<-[:GH_SyncedToEnvironment]-(:GH_SamlIdentityProvider)" + f"-[:GH_HasExternalIdentity]->(:GH_ExternalIdentity)" + f"-[:GH_MapsToUser]->(n) " + f"WHERE {composition_predicates} RETURN p" + ) + yield Edge( + kind=ek.SYNCED_TO_GH_USER, + start=ConditionalEdgePath( + kind=foreign_kind, + property_matchers=matchers, + ), + end=EdgePath(value=self.user.id, match_by="id"), + properties=GHEdgeProperties( + traversable=True, + composed=True, + query_composition=q, + ), ) - # # GH_MapsToUser → foreign IdP user node (match by name) - if foreign_kind and foreign_username: - match_with = PropertyMatch(key="name", value=foreign_username.upper()) - yield Edge( - kind=ek.MAPS_TO_USER, - start=EdgePath(value=self.node_id, match_by="id"), - end=ConditionalEdgePath( - kind="User", property_matchers=[match_with] - ), - properties=EdgeProperties(traversable=False), - ) - - # SyncedToGHUser: foreign IdP user → GitHub user (traversable, with composition) - if foreign_kind and foreign_username and self.node_id: - match_with = PropertyMatch(key="name", value=foreign_username.upper()) - - gh_id = self.node_id.upper() - q = ( - f"MATCH p=()<-[:GH_SyncedToEnvironment]-(:GH_SamlIdentityProvider)" - f"-[:GH_HasExternalIdentity]->(:GH_ExternalIdentity)" - f"-[:GH_MapsToUser]->(n) " - f"WHERE n.objectid = '{gh_id}' OR n.name = '{foreign_username.upper()}' RETURN p" - ) - yield Edge( - kind=ek.SYNCED_TO_GH_USER, - start=ConditionalEdgePath( - kind="User", property_matchers=[match_with] - ), - end=EdgePath(value=self.node_id, match_by="id"), - properties=GHEdgeProperties( - traversable=True, - composed=True, - query_composition=q, - ), - ) + @property + def service_provider_node_id(self) -> str | None: + return github_saml_service_provider_id( + self.idp.get("environment_type"), + self.environment_slug, + self.github_deployment_id, + ) + + @property + def saml_scoped_exact_match_values(self) -> list[str]: + if not self.saml_identity: + return [] + return saml_account_match_values( + self.saml_identity.username, + self.saml_identity.name_id, + ) + + @property + def enterprise_managed_user_scim_match_values(self) -> list[str]: + if self.idp.get("environment_type") != "enterprise": + return [] + if self.saml_scoped_exact_match_values or not self.scim_identity: + return [] + return saml_account_match_values(self.scim_identity.username) + + @property + def scoped_exact_match_values(self) -> list[str]: + return ( + self.saml_scoped_exact_match_values + or self.enterprise_managed_user_scim_match_values + ) + + @property + def entra_object_id_match_values(self) -> list[str]: + if not self.saml_identity: + return [] + return saml_attribute_match_values( + self.saml_identity.attributes, + ENTRA_OBJECT_ID_CLAIM, + ) + + @property + def saml_match_values(self) -> list[str]: + return saml_account_match_values( + *self.scoped_exact_match_values, + *self.entra_object_id_match_values, + ) + + @property + def saml_direct_binding_source(self) -> str | None: + if self.saml_scoped_exact_match_values: + return "GH_ExternalIdentity.saml_identity" + if self.enterprise_managed_user_scim_match_values: + return "GH_ExternalIdentity.scim_identity (Enterprise Managed Users)" + return None @property def edges(self): @@ -250,6 +333,27 @@ def edges(self): properties=EdgeProperties(traversable=False), ) + has_idp = bool(self.idp.get("id")) + service_provider_node_id = self.service_provider_node_id + match_values = self.saml_match_values + + if has_idp and service_provider_node_id and self.user and self.user.id and match_values: + yield Edge( + kind=ek.SAML_HAS_ACCOUNT, + start=EdgePath(value=service_provider_node_id, match_by="id"), + end=EdgePath(value=self.user.id, match_by="id"), + properties=SAMLHasAccountEdgeProperties( + traversable=False, + match_values=match_values, + scoped_exact_match_values=self.scoped_exact_match_values, + entra_object_id_match_values=self.entra_object_id_match_values, + direct_binding=True, + direct_binding_source=self.saml_direct_binding_source, + external_identity_id=self.node_id, + account_state="unknown", + ), + ) + # GH_MapsToUser → linked GitHub user node (match by id) if self.user and self.user.id: yield Edge( diff --git a/src/openhound_github/models/org_role.py b/src/openhound_github/models/org_role.py index 8547f5b..50b69e0 100644 --- a/src/openhound_github/models/org_role.py +++ b/src/openhound_github/models/org_role.py @@ -208,14 +208,22 @@ def as_node(self) -> GHNode: @property def _can_create_repos_edge(self): if self.type == "default" and (self.name == "owners" or self.name == "members"): + creation_flags = self._lookup.members_can_create_repository( + self.org_login + ) or (False, False, False, False) ( - members_can_create_repositories, - members_can_create_public_repositories, - members_can_create_internal_repositories, - members_can_create_private_repositories, - ) = self._lookup.members_can_create_repository(self.org_login) + can_create_repositories, + can_create_public_repositories, + can_create_internal_repositories, + can_create_private_repositories, + ) = creation_flags - if members_can_create_private_repositories: + if self.name == "owners": + can_create_repositories = True + can_create_public_repositories = True + can_create_private_repositories = True + + if can_create_private_repositories: yield Edge( kind=ek.CAN_CREATE_PRIVATE_REPOSITORIES, start=EdgePath(value=self.node_id, match_by="id"), @@ -223,7 +231,7 @@ def _can_create_repos_edge(self): properties=EdgeProperties(traversable=False), ) - if members_can_create_public_repositories: + if can_create_public_repositories: yield Edge( kind=ek.CAN_CREATE_PUBLIC_REPOSITORIES, start=EdgePath(value=self.node_id, match_by="id"), @@ -231,7 +239,7 @@ def _can_create_repos_edge(self): properties=EdgeProperties(traversable=False), ) - if members_can_create_internal_repositories: + if can_create_internal_repositories: yield Edge( kind=ek.CAN_CREATE_INTERNAL_REPOSITORIES, start=EdgePath(value=self.node_id, match_by="id"), @@ -239,7 +247,7 @@ def _can_create_repos_edge(self): properties=EdgeProperties(traversable=False), ) - if members_can_create_repositories: + if can_create_repositories: yield Edge( kind=ek.CAN_CREATE_REPOSITORIES, start=EdgePath(value=self.node_id, match_by="id"), diff --git a/src/openhound_github/models/org_secret.py b/src/openhound_github/models/org_secret.py index 495e7b8..6c79e18 100644 --- a/src/openhound_github/models/org_secret.py +++ b/src/openhound_github/models/org_secret.py @@ -12,6 +12,18 @@ from openhound_github.main import app +_ALL_REPOSITORY_CREATION_EDGE_KINDS = ( + ek.CAN_CREATE_REPOSITORIES, + ek.CAN_CREATE_PUBLIC_REPOSITORIES, + ek.CAN_CREATE_INTERNAL_REPOSITORIES, + ek.CAN_CREATE_PRIVATE_REPOSITORIES, +) +_PRIVATE_REPOSITORY_CREATION_EDGE_KINDS = ( + ek.CAN_CREATE_INTERNAL_REPOSITORIES, + ek.CAN_CREATE_PRIVATE_REPOSITORIES, +) + + @dataclass class GHOrgSecretProperties(GHNodeProperties): """Org secret properties and accordion panel queries. @@ -103,15 +115,35 @@ def as_node(self) -> GHNode: ), ) - def _read_secret_query(self, role_node_id: str) -> str: + @property + def _repository_creation_edge_kinds(self) -> tuple[str, ...]: + if self.visibility == "all": + return _ALL_REPOSITORY_CREATION_EDGE_KINDS + if self.visibility == "private": + return _PRIVATE_REPOSITORY_CREATION_EDGE_KINDS + return () + + def _read_secret_query( + self, role_node_id: str, edge_kinds: tuple[str, ...] + ) -> str: + creation_edges = "|".join(edge_kinds) return ( f"MATCH p=(:GH_OrgRole {{node_id:'{role_node_id}'}})" - f"-[:GH_CanCreateRepositories|GH_CanCreatePublicRepositories|" - f"GH_CanCreateInternalRepositories|GH_CanCreatePrivateRepositories]->" + f"-[:{creation_edges}]->" f"(:GH_Organization)-[:GH_Contains]->" f"(:GH_OrgSecret {{node_id:'{self.node_id}'}}) RETURN p" ) + def _members_can_create_repository_in_scope( + self, edge_kinds: tuple[str, ...] + ) -> bool: + creation_flags = self._lookup.members_can_create_repository(self.org_login) + if not creation_flags: + return False + + permissions = dict(zip(_ALL_REPOSITORY_CREATION_EDGE_KINDS, creation_flags)) + return any(bool(permissions.get(edge_kind)) for edge_kind in edge_kinds) + @property def _all_repo_edges(self): if self.visibility == "all": @@ -149,7 +181,8 @@ def _contains_edge(self): @property def _composed_read_secret_edges(self): - if self.visibility != "all": + edge_kinds = self._repository_creation_edge_kinds + if not edge_kinds: return owners_role_id = f"{self.org_node_id}_owners" @@ -160,11 +193,13 @@ def _composed_read_secret_edges(self): properties=GHEdgeProperties( traversable=True, composed=True, - query_composition=self._read_secret_query(owners_role_id), + query_composition=self._read_secret_query( + owners_role_id, edge_kinds + ), ), ) - if self._lookup.members_can_create_repository(self.org_login): + if self._members_can_create_repository_in_scope(edge_kinds): members_role_id = f"{self.org_node_id}_members" yield Edge( kind=ek.CAN_READ_SECRET, @@ -173,7 +208,9 @@ def _composed_read_secret_edges(self): properties=GHEdgeProperties( traversable=True, composed=True, - query_composition=self._read_secret_query(members_role_id), + query_composition=self._read_secret_query( + members_role_id, edge_kinds + ), ), ) diff --git a/src/openhound_github/models/org_variable.py b/src/openhound_github/models/org_variable.py index 708b55e..1cdcdef 100644 --- a/src/openhound_github/models/org_variable.py +++ b/src/openhound_github/models/org_variable.py @@ -48,6 +48,13 @@ class GHOrgVariableProperties(GHNodeProperties): description="Org contains variable", traversable=False, ), + EdgeDef( + start=nk.REPOSITORY, + end=nk.ORG_VARIABLE, + kind=ek.HAS_VARIABLE, + description="Repository can access org variable", + traversable=True, + ), ], ) class OrgVariable(BaseAsset): @@ -91,17 +98,46 @@ def as_node(self) -> GHNode: ), ) + @property + def _all_repo_edges(self): + if self.visibility == "all": + for repo in self._lookup.repository_node_ids_for_org(self.org_login): + for repo_node_id in repo: + yield Edge( + kind=ek.HAS_VARIABLE, + start=EdgePath(value=repo_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) + + @property + def _private_repo_edges(self): + if self.visibility == "private": + for repo in self._lookup.private_repository_node_ids_for_org( + self.org_login + ): + for repo_node_id in repo: + yield Edge( + kind=ek.HAS_VARIABLE, + start=EdgePath(value=repo_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) + + @property + def _contains_edge(self): + yield Edge( + kind=ek.CONTAINS, + start=EdgePath(value=self.org_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=EdgeProperties(traversable=False), + ) + @property def edges(self): - # TODO: Check if this should indeed not return CONTAINS edge - return [] - # yield Edge( - # kind=ek.CONTAINS, - # start=EdgePath(value=self._lookup.org_id(), match_by="id"), - # end=EdgePath(value=self.node_id, match_by="id"), - # properties=EdgeProperties(traversable=False), - # ) - # + yield from self._contains_edge + yield from self._all_repo_edges + yield from self._private_repo_edges @app.asset( @@ -111,7 +147,7 @@ def edges(self): end=nk.ORG_VARIABLE, kind=ek.HAS_VARIABLE, description="Repository can access org variable", - traversable=False, + traversable=True, ) ], ) @@ -140,5 +176,5 @@ def edges(self): kind=ek.HAS_VARIABLE, start=EdgePath(value=self.repository_node_id, match_by="id"), end=EdgePath(value=self.node_id, match_by="id"), - properties=EdgeProperties(traversable=False), + properties=EdgeProperties(traversable=True), ) diff --git a/src/openhound_github/models/repository.py b/src/openhound_github/models/repository.py index c217eb6..49fafe7 100644 --- a/src/openhound_github/models/repository.py +++ b/src/openhound_github/models/repository.py @@ -247,14 +247,14 @@ def as_node(self) -> GHNode: actions_enabled=self.actions_enabled, self_hosted_runners_enabled=self.self_hosted_runners_enabled, # secret_scanning=self.secret_scanning, - query_branches=f"MATCH p=(:GH_Repository {{node_id: '{rid}'}})-[:GH_HasBranch]->(:GH_Branch) RETURN p", - query_protected_branches=f"MATCH p=(:GH_Repository {{node_id: '{rid}'}})-[:GH_HasBranch]->(:GH_Branch)<-[:GH_ProtectedBy]-(:GH_BranchProtectionRule) RETURN p", + query_branches=f"MATCH p=(:GH_Repository {{node_id: '{rid}'}})-[:GH_Contains]->(:GH_Branch) RETURN p", + query_protected_branches=f"MATCH p=(:GH_Repository {{node_id: '{rid}'}})-[:GH_Contains]->(:GH_Branch)<-[:GH_ProtectedBy]-(:GH_BranchProtectionRule) RETURN p", query_branch_protection_rules=f"MATCH p=(:GH_Repository {{node_id: '{rid}'}})-[:GH_Contains]->(:GH_BranchProtectionRule) RETURN p", query_roles=f"MATCH p=(:GH_RepoRole)-[*1..]->(:GH_Repository {{node_id: '{rid}'}}) RETURN p", query_teams=f"MATCH p=(:GH_Team)-[:GH_MemberOf|GH_HasRole*1..]->(:GH_RepoRole)-[]->(:GH_Repository {{node_id: '{rid}'}}) RETURN p", - query_workflows=f"MATCH p=(:GH_Repository {{node_id:'{rid}'}})-[:GH_HasWorkflow]->(:GH_Workflow)-[:GH_HasJob]->(:GH_WorkflowJob)-[:GH_HasStep]->(step:GH_WorkflowStep) OPTIONAL MATCH p1=(step)-[:GH_UsesSecret]->(:GH_Secret) OPTIONAL MATCH p2=(step)-[:GH_UsesVariable]->(:GH_Variable) RETURN p,p1,p2", + query_workflows=f"MATCH p=(:GH_Repository {{node_id:'{rid}'}})-[:GH_Contains]->(:GH_Workflow)-[:GH_Contains]->(:GH_WorkflowJob)-[:GH_Contains]->(step:GH_WorkflowStep) OPTIONAL MATCH p1=(step)-[:GH_UsesSecret]->(:GH_Secret) OPTIONAL MATCH p2=(step)-[:GH_UsesVariable]->(:GH_Variable) RETURN p,p1,p2", query_runners=f"MATCH p=(:GH_Repository {{node_id:'{rid}'}})-[:GH_CanUseRunner]->(:GH_Runner) RETURN p", - query_environments=f"MATCH p=(:GH_Repository {{node_id: '{rid}'}})-[:GH_HasEnvironment]->(:GH_Environment) RETURN p", + query_environments=f"MATCH p=(:GH_Repository {{node_id: '{rid}'}})-[:GH_Contains]->(:GH_Environment) RETURN p", query_secrets=f"MATCH p=(:GH_Repository {{node_id:'{rid}'}})-[:GH_HasSecret]->(:GH_Secret) RETURN p", query_variables=f"MATCH p=(:GH_Repository {{node_id:'{rid}'}})-[:GH_HasVariable]->(:GH_Variable) RETURN p", query_secret_scanning_alerts=f"MATCH p=(:GH_Repository {{node_id:'{rid}'}})-[:GH_Contains]->(:GH_SecretScanningAlert) RETURN p", diff --git a/src/openhound_github/models/repository_role.py b/src/openhound_github/models/repository_role.py index 2e9d0a7..9436064 100644 --- a/src/openhound_github/models/repository_role.py +++ b/src/openhound_github/models/repository_role.py @@ -666,6 +666,13 @@ class GHRepoRoleProperties(GHNodeProperties): description="Role can delete discussion comments", traversable=False, ), + EdgeDef( + start=nk.REPO_ROLE, + end=nk.REPOSITORY, + kind=ek.CAN_CREATE_ENVIRONMENT, + description="Role can create new environments in the repository by editing workflows", + traversable=True, + ), ], ) class RepoRole(BaseAsset): @@ -909,7 +916,19 @@ def _can_create_branch_edges(self): kind=ek.CAN_CREATE_BRANCH, start=EdgePath(value=self.node_id, match_by="id"), end=EdgePath(value=self.repository_node_id, match_by="id"), - properties=EdgeProperties(traversable=False), + properties=EdgeProperties(traversable=True), + ) + + @property + def _can_create_environment_edges(self): + write_roles = ["write", "maintain", "admin"] + if self.name in write_roles or self.base_role in write_roles: + if self._lookup.role_can_create_branch(self.id, self.repository_node_id): + yield Edge( + kind=ek.CAN_CREATE_ENVIRONMENT, + start=EdgePath(value=self.node_id, match_by="id"), + end=EdgePath(value=self.repository_node_id, match_by="id"), + properties=EdgeProperties(traversable=True), ) @property @@ -965,5 +984,6 @@ def edges(self): yield from self._bypass_branch_protection_edges yield from self._bypass_push_protected_branch_edges yield from self._can_create_branch_edges + yield from self._can_create_environment_edges yield from self._can_edit_repo_protection_edges yield from self._can_edit_branch_protection_edges diff --git a/src/openhound_github/models/saml_assertion_consumer_service.py b/src/openhound_github/models/saml_assertion_consumer_service.py new file mode 100644 index 0000000..26455ca --- /dev/null +++ b/src/openhound_github/models/saml_assertion_consumer_service.py @@ -0,0 +1,126 @@ +from dataclasses import dataclass + +from openhound.core.asset import BaseAsset, EdgeDef, NodeDef +from openhound.core.models.entries_dataclass import Edge, EdgePath +from openhound_github.graph import GHNode, GHNodeProperties +from openhound_github.kinds import edges as ek +from openhound_github.kinds import nodes as nk +from openhound_github.main import app + +from .saml_helpers import ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + DEFAULT_GITHUB_WEB_ORIGIN, + SAML_CONTRACT_VERSION, + SAMLRelationshipEdgeProperties, + github_saml_acs_id, + github_saml_route, + github_saml_service_provider_id, + normalize_scope_type, +) + +@dataclass +class SAMLAssertionConsumerServiceProperties(GHNodeProperties): + """Properties for a normalized GitHub ACS route. + + Attributes: + native_id: The GitHub enterprise or organization node ID. + scope_type: The GitHub SAML scope type. + scope_slug: The GitHub enterprise or organization slug. + acs_url: The byte-exact GitHub ACS URL. + sp_entity_id: The byte-exact GitHub service provider entity ID. + github_deployment_id: Stable GitHub deployment identifier. + github_web_origin: Browser origin for the GitHub deployment. + route_source: Convention used to derive the ACS route. + schema_contract_version: Shared OpenGraph SAML contract version. + """ + native_id: str | None = None + scope_type: str | None = None + scope_slug: str | None = None + acs_url: str | None = None + sp_entity_id: str | None = None + github_deployment_id: str | None = None + github_web_origin: str | None = None + route_source: str | None = None + schema_contract_version: str = SAML_CONTRACT_VERSION + +@app.asset( + node=NodeDef( + kind=nk.SAML_ASSERTION_CONSUMER_SERVICE, + description="Normalized SAML assertion consumer service derived from GitHub SAML configuration", + icon="id-badge", + properties=SAMLAssertionConsumerServiceProperties, + ), + edges=[ + EdgeDef( + start=nk.SAML_SERVICE_PROVIDER, + end=nk.SAML_ASSERTION_CONSUMER_SERVICE, + kind=ek.SAML_HAS_ASSERTION_CONSUMER_SERVICE, + description="Normalized SAML service provider has this assertion consumer service", + traversable=False, + ), + ], +) +class SamlAssertionConsumerService(BaseAsset): + environment_slug: str + environment_type: str + environment_node_id: str | None = None + environment_name: str | None = None + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN + + @property + def service_provider_node_id(self) -> str: + return github_saml_service_provider_id( + self.environment_type, + self.environment_slug, + self.github_deployment_id, + ) + + @property + def saml_route(self) -> tuple[str, str]: + return github_saml_route( + self.environment_type, + self.environment_slug, + self.github_web_origin, + ) + + @property + def node_id(self) -> str: + return github_saml_acs_id( + self.environment_type, + self.environment_slug, + self.github_deployment_id, + ) + + @property + def as_node(self) -> GHNode: + acs_url, sp_entity_id = self.saml_route + scope_type = normalize_scope_type(self.environment_type) + + return GHNode( + kinds=[nk.SAML_ASSERTION_CONSUMER_SERVICE], + properties=SAMLAssertionConsumerServiceProperties( + name=self.node_id, + displayname=acs_url, + node_id=self.node_id, + acs_url=acs_url, + sp_entity_id=sp_entity_id, + environmentid=self.environment_node_id, + native_id=self.environment_node_id, + scope_type=scope_type, + scope_slug=self.environment_slug, + github_deployment_id=self.github_deployment_id, + github_web_origin=self.github_web_origin, + route_source=f"github_{scope_type}_scope_convention", + schema_contract_version=SAML_CONTRACT_VERSION, + ), + ) + + @property + def edges(self): + yield Edge( + kind=ek.SAML_HAS_ASSERTION_CONSUMER_SERVICE, + start=EdgePath(value=self.service_provider_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=SAMLRelationshipEdgeProperties(traversable=False), + ) diff --git a/src/openhound_github/models/saml_helpers.py b/src/openhound_github/models/saml_helpers.py new file mode 100644 index 0000000..7e8691a --- /dev/null +++ b/src/openhound_github/models/saml_helpers.py @@ -0,0 +1,332 @@ +from collections.abc import Mapping +from dataclasses import dataclass +from typing import Any +from urllib.parse import quote, urlparse + +from openhound.core.models.entries_dataclass import EdgeProperties, PropertyMatch + +from openhound_github.kinds import nodes as nk + + +SAML_CONTRACT_VERSION = "opengraph-saml-v0.3.0" +ENTRA_OBJECT_ID_CLAIM = "http://schemas.microsoft.com/identity/claims/objectidentifier" +ENTRA_TENANT_ID_CLAIM = "http://schemas.microsoft.com/identity/claims/tenantid" +DEFAULT_GITHUB_DEPLOYMENT_ID = "github.com" +DEFAULT_GITHUB_WEB_ORIGIN = "https://github.com" + +_FOREIGN_USER_KIND = { + "entra": nk.AZ_USER, + "okta": nk.OKTA_USER, + "pingone": nk.PINGONE_USER, +} +_FOREIGN_USER_ENVIRONMENT_PROPERTY = { + nk.OKTA_USER: "tenant_domain", + nk.PINGONE_USER: "environmentid", +} + + +@dataclass +class SAMLRelationshipEdgeProperties(EdgeProperties): + schema_contract_version: str = SAML_CONTRACT_VERSION + + +def _clean(value: Any) -> str | None: + if value is None: + return None + text = str(value).strip() + return text or None + + +def _dedupe(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + cleaned = _clean(value) + if cleaned and cleaned not in seen: + seen.add(cleaned) + result.append(cleaned) + return result + + +def detect_foreign_idp( + issuer: str | None, sso_url: str | None +) -> tuple[str | None, str | None]: + """Return the canonical foreign IdP type and tenant/environment identifier.""" + if not issuer: + return None, None + + if issuer.startswith("https://auth.pingone.com/"): + return "pingone", issuer.split("/")[3] + + if issuer.startswith("https://sts.windows.net/"): + return "entra", issuer.split("/")[3] + + if issuer.startswith("http://www.okta.com/"): + domain = urlparse(sso_url).netloc if sso_url else None + return "okta", domain + + return None, None + +def foreign_user_kind(foreign_idp_type: str | None) -> str: + return _FOREIGN_USER_KIND.get(foreign_idp_type or "", "") + + +def foreign_user_matchers( + foreign_kind: str | None, + foreign_environment_id: str | None, + foreign_username: str | None, + saml_attributes: list[Any] | None = None, +) -> list[PropertyMatch]: + """Return tenant-scoped matchers for foreign IdP user correlation.""" + if foreign_kind == nk.AZ_USER: + tenant_ids = saml_attribute_match_values( + saml_attributes or [], + ENTRA_TENANT_ID_CLAIM, + ) + object_ids = saml_attribute_match_values( + saml_attributes or [], + ENTRA_OBJECT_ID_CLAIM, + ) + if ( + not foreign_environment_id + or len(tenant_ids) != 1 + or len(object_ids) != 1 + or tenant_ids[0].casefold() != foreign_environment_id.casefold() + ): + return [] + return [ + PropertyMatch(key="tenantid", value=tenant_ids[0].upper()), + PropertyMatch(key="objectid", value=object_ids[0].upper()), + ] + + environment_property = _FOREIGN_USER_ENVIRONMENT_PROPERTY.get(foreign_kind or "") + if not environment_property or not foreign_environment_id or not foreign_username: + return [] + return [ + PropertyMatch(key=environment_property, value=foreign_environment_id), + PropertyMatch(key="name", value=foreign_username.upper()), + ] + + +def github_deployment_context(host: str) -> tuple[str, str]: + """Return a stable deployment ID and browser origin for a GitHub API host.""" + parsed = urlparse(host) + if not parsed.scheme or not parsed.hostname: + raise ValueError("GitHub host must be an absolute HTTP(S) URL") + if parsed.scheme not in {"http", "https"}: + raise ValueError("GitHub host must use HTTP or HTTPS") + + hostname = parsed.hostname.lower() + if hostname in {"api.github.com", "github.com"}: + return DEFAULT_GITHUB_DEPLOYMENT_ID, DEFAULT_GITHUB_WEB_ORIGIN + + authority = hostname + if parsed.port: + authority = f"{authority}:{parsed.port}" + return authority, f"{parsed.scheme.lower()}://{authority}" + + +def _saml_id( + resource: str, + scope_type: str, + scope_slug: str, + github_deployment_id: str, +) -> str: + deployment_id = github_deployment_id.strip().lower() + if deployment_id == DEFAULT_GITHUB_DEPLOYMENT_ID: + return f"github:saml:{resource}:{scope_type}:{scope_slug}" + encoded_deployment = quote(deployment_id, safe=".-_") + return f"github:{encoded_deployment}:saml:{resource}:{scope_type}:{scope_slug}" + + +def github_enterprise_saml_service_provider_id( + slug: str, + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID, +) -> str: + return _saml_id("sp", "enterprise", slug, github_deployment_id) + + +def github_enterprise_saml_issuer_id( + slug: str, + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID, +) -> str: + return _saml_id("trusted-issuer", "enterprise", slug, github_deployment_id) + + +def github_enterprise_saml_acs_id( + slug: str, + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID, +) -> str: + return _saml_id("acs", "enterprise", slug, github_deployment_id) + + +def github_enterprise_acs_url( + slug: str, + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN, +) -> str: + return f"{github_web_origin.rstrip('/')}/enterprises/{slug}/saml/consume" + + +def github_enterprise_sp_entity_id( + slug: str, + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN, +) -> str: + return f"{github_web_origin.rstrip('/')}/enterprises/{slug}" + + +def github_org_saml_service_provider_id( + login: str, + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID, +) -> str: + return _saml_id("sp", "org", login, github_deployment_id) + + +def github_org_saml_issuer_id( + login: str, + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID, +) -> str: + return _saml_id("trusted-issuer", "org", login, github_deployment_id) + + +def github_org_saml_acs_id( + login: str, + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID, +) -> str: + return _saml_id("acs", "org", login, github_deployment_id) + + +def github_org_acs_url( + login: str, + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN, +) -> str: + return f"{github_web_origin.rstrip('/')}/orgs/{login}/saml/consume" + + +def github_org_sp_entity_id( + login: str, + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN, +) -> str: + return f"{github_web_origin.rstrip('/')}/orgs/{login}" + + +def github_saml_service_provider_id( + environment_type: str | None, + environment_slug: str | None, + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID, +) -> str | None: + if not environment_type or not environment_slug: + return None + if environment_type == "enterprise": + return github_enterprise_saml_service_provider_id( + environment_slug, + github_deployment_id, + ) + return github_org_saml_service_provider_id( + environment_slug, + github_deployment_id, + ) + + +def github_saml_issuer_id( + environment_type: str | None, + environment_slug: str | None, + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID, +) -> str | None: + if not environment_type or not environment_slug: + return None + if environment_type == "enterprise": + return github_enterprise_saml_issuer_id( + environment_slug, + github_deployment_id, + ) + return github_org_saml_issuer_id( + environment_slug, + github_deployment_id, + ) + + +def github_saml_acs_id( + environment_type: str | None, + environment_slug: str | None, + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID, +) -> str | None: + if not environment_type or not environment_slug: + return None + if environment_type == "enterprise": + return github_enterprise_saml_acs_id( + environment_slug, + github_deployment_id, + ) + return github_org_saml_acs_id( + environment_slug, + github_deployment_id, + ) + + +def github_saml_route( + environment_type: str, + environment_slug: str, + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN, +) -> tuple[str, str]: + if environment_type == "enterprise": + return ( + github_enterprise_acs_url(environment_slug, github_web_origin), + github_enterprise_sp_entity_id(environment_slug, github_web_origin), + ) + return ( + github_org_acs_url(environment_slug, github_web_origin), + github_org_sp_entity_id(environment_slug, github_web_origin), + ) + + +def saml_account_match_values(*values: str | None) -> list[str]: + """Return source-exact SAML account values with blank values removed.""" + return _dedupe(list(values)) + + +def saml_attribute_match_values( + attributes: list[Any], attribute_name: str +) -> list[str]: + """Return source-exact values for one explicitly named SAML attribute.""" + values: list[str | None] = [] + for attribute in attributes: + if isinstance(attribute, Mapping): + name = attribute.get("name") + value = attribute.get("value") + else: + name = getattr(attribute, "name", None) + value = getattr(attribute, "value", None) + if name == attribute_name: + values.append(value) + return _dedupe(values) + + +def build_service_provider_node_id( + environment_type: str | None, + environment_slug: str | None, + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID, +) -> str | None: + return github_saml_service_provider_id( + environment_type, + environment_slug, + github_deployment_id, + ) + +def build_issuer_node_id(issuer: str | None) -> str | None: + if not issuer: + return None + return f"saml:trusted-issuer:{issuer}" + +def build_saml_route( + environment_type: str, + environment_slug: str, + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN, +) -> tuple[str, str]: + return github_saml_route( + environment_type, + environment_slug, + github_web_origin, + ) + +def normalize_scope_type(environment_type: str) -> str: + return "organization" if environment_type == "org" else environment_type diff --git a/src/openhound_github/models/saml_issuer.py b/src/openhound_github/models/saml_issuer.py new file mode 100644 index 0000000..bd01dcd --- /dev/null +++ b/src/openhound_github/models/saml_issuer.py @@ -0,0 +1,117 @@ +from dataclasses import dataclass + +from openhound.core.asset import BaseAsset, NodeDef, EdgeDef +from openhound.core.models.entries_dataclass import Edge, EdgePath +from openhound_github.graph import GHNode, GHNodeProperties +from openhound_github.kinds import nodes as nk +from openhound_github.kinds import edges as ek +from openhound_github.main import app + +from .saml_helpers import ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + DEFAULT_GITHUB_WEB_ORIGIN, + SAML_CONTRACT_VERSION, + SAMLRelationshipEdgeProperties, + github_saml_issuer_id, + github_saml_service_provider_id, + normalize_scope_type, +) + +@dataclass +class SAMLIssuerProperties(GHNodeProperties): + """Properties for a normalized GitHub trusted SAML issuer. + + Attributes: + native_id: The GitHub enterprise or organization node ID. + scope_type: The GitHub SAML scope type. + scope_slug: The GitHub enterprise or organization slug. + entity_id: The trusted SAML issuer entity ID. + github_deployment_id: Stable GitHub deployment identifier. + github_web_origin: Browser origin for the GitHub deployment. + native_source_field: Native GitHub field that supplied the issuer. + schema_contract_version: Shared OpenGraph SAML contract version. + """ + native_id: str | None = None + scope_type: str | None = None + scope_slug: str | None = None + entity_id: str | None = None + github_deployment_id: str | None = None + github_web_origin: str | None = None + native_source_field: str | None = None + schema_contract_version: str = SAML_CONTRACT_VERSION + +@app.asset( + node=NodeDef( + kind=nk.SAML_ISSUER, + description="Normalized SAML issuer derived from GitHub SAML configuration", + icon="id-badge", + properties=SAMLIssuerProperties, + ), + edges=[ + EdgeDef( + start=nk.SAML_SERVICE_PROVIDER, + end=nk.SAML_ISSUER, + kind=ek.SAML_TRUSTS_ISSUER, + description="Normalized SAML service provider trusts this issuer", + traversable=False, + ), + ], +) +class SamlIssuer(BaseAsset): + issuer: str + environment_slug: str + environment_type: str + environment_node_id: str | None = None + environment_name: str | None = None + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN + + @property + def node_id(self) -> str | None: + if not self.issuer: + return None + return github_saml_issuer_id( + self.environment_type, + self.environment_slug, + self.github_deployment_id, + ) + + @property + def service_provider_node_id(self) -> str: + return github_saml_service_provider_id( + self.environment_type, + self.environment_slug, + self.github_deployment_id, + ) + + @property + def as_node(self) -> GHNode: + scope_type = normalize_scope_type(self.environment_type) + + return GHNode( + kinds=[nk.SAML_ISSUER], + properties=SAMLIssuerProperties( + name=self.node_id, + displayname=self.issuer, + node_id=self.node_id, + entity_id=self.issuer, + environmentid=self.environment_node_id, + native_id=self.environment_node_id, + scope_type=scope_type, + scope_slug=self.environment_slug, + github_deployment_id=self.github_deployment_id, + github_web_origin=self.github_web_origin, + native_source_field="GH_SamlIdentityProvider.issuer", + schema_contract_version=SAML_CONTRACT_VERSION, + ), + ) + + @property + def edges(self): + if self.issuer: + yield Edge( + kind=ek.SAML_TRUSTS_ISSUER, + start=EdgePath(value=self.service_provider_node_id, match_by="id"), + end=EdgePath(value=self.node_id, match_by="id"), + properties=SAMLRelationshipEdgeProperties(traversable=False), + ) diff --git a/src/openhound_github/models/saml_provider.py b/src/openhound_github/models/saml_provider.py index 809af03..d981c01 100644 --- a/src/openhound_github/models/saml_provider.py +++ b/src/openhound_github/models/saml_provider.py @@ -2,12 +2,17 @@ from openhound.core.asset import BaseAsset, EdgeDef, NodeDef from openhound.core.models.entries_dataclass import Edge, EdgePath, EdgeProperties -from pydantic import Field +from pydantic import ConfigDict, Field from openhound_github.graph import GHNode, GHNodeProperties from openhound_github.kinds import edges as ek from openhound_github.kinds import nodes as nk from openhound_github.main import app +from .saml_helpers import ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + DEFAULT_GITHUB_WEB_ORIGIN, + detect_foreign_idp, +) @dataclass @@ -33,6 +38,8 @@ class GHSamlProviderProperties(GHNodeProperties): idp_certificate: str | None = None environment_name: str | None = None foreign_environment_id: str | None = None + github_deployment_id: str | None = None + github_web_origin: str | None = None query_environments: str | None = None query_external_identities: str | None = None @@ -40,7 +47,7 @@ class GHSamlProviderProperties(GHNodeProperties): @app.asset( node=NodeDef( kind=nk.SAML_IDENTITY_PROVIDER, - description="GitHub Organization SAML Identity Provider", + description="SAML Identity Provider for GitHub Organization or Enterprise Account", icon="id-badge", properties=GHSamlProviderProperties, ), @@ -49,7 +56,14 @@ class GHSamlProviderProperties(GHNodeProperties): start=nk.ORGANIZATION, end=nk.SAML_IDENTITY_PROVIDER, kind=ek.HAS_SAML_IDENTITY_PROVIDER, - description="Org uses this SAML IdP", + description="GitHub Organization uses this SAML IdP", + traversable=False, + ), + EdgeDef( + start=nk.ENTERPRISE, + end=nk.SAML_IDENTITY_PROVIDER, + kind=ek.HAS_SAML_IDENTITY_PROVIDER, + description="GitHub Enterprise Account uses this SAML IdP", traversable=False, ), ], @@ -57,6 +71,8 @@ class GHSamlProviderProperties(GHNodeProperties): class SamlProvider(BaseAsset): """One record from `saml_provider` → one GH_SamlIdentityProvider node + GH_HasSamlIdentityProvider from org.""" + model_config = ConfigDict(populate_by_name=True) + id: str digest_method: str | None = None @@ -69,9 +85,12 @@ class SamlProvider(BaseAsset): foreign_environment_id: str | None = None # tenant/org ID in the foreign IdP # Additional - org_login: str - org_name: str - org_node_id: str # organization.id (GraphQL global ID) + environment_node_id: str # organization.id (GraphQL global ID) + environment_name: str + environment_slug: str + environment_type: str + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN @property def node_id(self) -> str: @@ -80,23 +99,25 @@ def node_id(self) -> str: @property def as_node(self) -> GHNode: - iid = self.node_id + _, foreign_environment_id = detect_foreign_idp(self.issuer, self.sso_url) return GHNode( kinds=[nk.SAML_IDENTITY_PROVIDER], properties=GHSamlProviderProperties( - name=self.org_name, - displayname=self.org_name, - node_id=iid, + name=self.node_id, + displayname=self.environment_name, + node_id=self.node_id, issuer=self.issuer, sso_url=self.sso_url, signature_method=self.signature_method, digest_method=self.digest_method, idp_certificate=self.idp_certificate, - environment_name=self.org_name, - environmentid=self.org_node_id, - foreign_environment_id=self.foreign_environment_id, - query_environments=f"MATCH p=(:GH_SamlIdentityProvider {{node_id:'{iid}'}})<-[:GH_HasSamlIdentityProvider]-(:GH_Organization) RETURN p", - query_external_identities=f"MATCH p=(:GH_SamlIdentityProvider {{node_id:'{iid}'}})-[:GH_HasExternalIdentity]->() RETURN p", + environment_name=self.environment_name, + environmentid=self.environment_node_id, + foreign_environment_id=foreign_environment_id, + github_deployment_id=self.github_deployment_id, + github_web_origin=self.github_web_origin, + query_environments=f"MATCH p=(:GitHub)-[:GH_HasSamlIdentityProvider]->(:GH_SamlIdentityProvider {{node_id:'{self.node_id}'}}) RETURN p", + query_external_identities=f"MATCH p=(:GH_SamlIdentityProvider {{node_id:'{self.node_id}'}})-[:GH_HasExternalIdentity]->(:GH_ExternalIdentity) RETURN p", ), ) @@ -104,7 +125,7 @@ def as_node(self) -> GHNode: def edges(self): yield Edge( kind=ek.HAS_SAML_IDENTITY_PROVIDER, - start=EdgePath(value=self.org_node_id, match_by="id"), + start=EdgePath(value=self.environment_node_id, match_by="id"), end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), ) diff --git a/src/openhound_github/models/saml_service_provider.py b/src/openhound_github/models/saml_service_provider.py new file mode 100644 index 0000000..661feff --- /dev/null +++ b/src/openhound_github/models/saml_service_provider.py @@ -0,0 +1,134 @@ +from dataclasses import dataclass + +from openhound.core.asset import BaseAsset, EdgeDef, NodeDef +from openhound.core.models.entries_dataclass import Edge, EdgePath + +from openhound_github.graph import GHNode, GHNodeProperties +from openhound_github.kinds import edges as ek +from openhound_github.kinds import nodes as nk +from openhound_github.main import app +from .saml_helpers import ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + DEFAULT_GITHUB_WEB_ORIGIN, + SAML_CONTRACT_VERSION, + SAMLRelationshipEdgeProperties, + github_saml_issuer_id, + github_saml_route, + github_saml_service_provider_id, + normalize_scope_type, +) + +@dataclass +class SAMLServiceProviderProperties(GHNodeProperties): + """Properties for a normalized GitHub SAML service provider. + + Attributes: + native_id: The GitHub enterprise or organization node ID. + scope_type: The GitHub SAML scope type. + scope_slug: The GitHub enterprise or organization slug. + saml_provider_id: The native GitHub SAML provider node ID. + enabled: Whether SAML is enabled for the scope. + entity_id: The SAML service provider entity ID. + environment_name: The name of the GitHub organization or enterprise. + github_deployment_id: Stable GitHub deployment identifier. + github_web_origin: Browser origin for the GitHub deployment. + schema_contract_version: Shared OpenGraph SAML contract version. + """ + + native_id: str | None = None + scope_type: str | None = None + scope_slug: str | None = None + saml_provider_id: str | None = None + enabled: bool | None = None + entity_id: str | None = None + environment_name: str | None = None + github_deployment_id: str | None = None + github_web_origin: str | None = None + schema_contract_version: str = SAML_CONTRACT_VERSION + +@app.asset( + node=NodeDef( + kind=nk.SAML_SERVICE_PROVIDER, + description="Normalized SAML service provider derived from GitHub SAML configuration", + icon="id-badge", + properties=SAMLServiceProviderProperties, + ), + edges=[ + EdgeDef( + start=nk.SAML_IDENTITY_PROVIDER, + end=nk.SAML_SERVICE_PROVIDER, + kind=ek.SAML_IMPLEMENTS, + description="GitHub native SAML configuration implements a normalized SAML service provider", + traversable=False, + ), + ], +) + +class SamlServiceProvider(BaseAsset): + id: str + issuer: str | None = None + environment_node_id: str + environment_name: str + environment_slug: str + environment_type: str + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN + + @property + def service_provider_node_id(self) -> str: + return github_saml_service_provider_id( + self.environment_type, + self.environment_slug, + self.github_deployment_id, + ) + + @property + def issuer_node_id(self) -> str | None: + if not self.issuer: + return None + return github_saml_issuer_id( + self.environment_type, + self.environment_slug, + self.github_deployment_id, + ) + + @property + def saml_route(self) -> tuple[str, str]: + return github_saml_route( + self.environment_type, + self.environment_slug, + self.github_web_origin, + ) + + @property + def as_node(self) -> GHNode: + _, sp_entity_id = self.saml_route + scope_type = normalize_scope_type(self.environment_type) + return GHNode( + kinds=[nk.SAML_SERVICE_PROVIDER], + properties=SAMLServiceProviderProperties( + name=self.service_provider_node_id, + displayname=self.environment_name, + node_id=self.service_provider_node_id, + entity_id=sp_entity_id, + environment_name=self.environment_name, + environmentid=self.environment_node_id, + native_id=self.environment_node_id, + scope_type=scope_type, + scope_slug=self.environment_slug, + saml_provider_id=self.id, + enabled=True, + github_deployment_id=self.github_deployment_id, + github_web_origin=self.github_web_origin, + schema_contract_version=SAML_CONTRACT_VERSION, + ), + ) + + @property + def edges(self): + yield Edge( + kind=ek.SAML_IMPLEMENTS, + start=EdgePath(value=self.id, match_by="id"), + end=EdgePath(value=self.service_provider_node_id, match_by="id"), + properties=SAMLRelationshipEdgeProperties(traversable=False), + ) diff --git a/src/openhound_github/models/scim_user.py b/src/openhound_github/models/scim_user.py index 5fd988d..3151c7d 100644 --- a/src/openhound_github/models/scim_user.py +++ b/src/openhound_github/models/scim_user.py @@ -1,25 +1,337 @@ -from typing import Optional +"""Normalized SCIM graph assets emitted from GitHub SCIM APIs.""" +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any, ClassVar + +from dlt.common.libs.pydantic import DltConfig +from openhound.core.asset import BaseAsset, EdgeDef, NodeDef +from openhound.core.models.entries_dataclass import ( + ConditionalEdgePath, + Edge, + EdgePath, + EdgeProperties, + Node, + NodeProperties, + PropertyMatch, +) from pydantic import BaseModel, ConfigDict, Field +from openhound_github.kinds import edges as ek +from openhound_github.kinds import nodes as nk +from openhound_github.main import app +from openhound_github.models.saml_helpers import detect_foreign_idp + + +def scim_organization_id(scope_node_id: str) -> str: + return f"SCIM_Organization_{scope_node_id}" + + +@dataclass +class ScimNodeProperties(NodeProperties): + collected: bool = True + external_id: str | None = None + user_name: str | None = None + enabled: bool | None = None + given_name: str | None = None + family_name: str | None = None + mail: str | None = None + profile_url: str | None = None + enterprise: str | None = None + organization: str | None = None + source_kind: str | None = None + + +@dataclass +class ScimNode(Node): + id: str + + def __post_init__(self): + return None + class Name(BaseModel): + model_config = ConfigDict(populate_by_name=True) + given_name: str | None = Field(default=None, alias="givenName") family_name: str | None = Field(default=None, alias="familyName") formatted: str | None = None -class ScimResource(BaseModel): - model_config = ConfigDict(populate_by_name=True) +class ScimMeta(BaseModel): + model_config = ConfigDict(extra="allow", populate_by_name=True) + + created: str | None = None + last_modified: str | None = Field(default=None, alias="lastModified") + location: str | None = None + + +def _primary_email(emails: list[dict[str, Any]]) -> str | None: + for email in emails: + if email.get("primary") is True and email.get("value"): + return str(email["value"]) + for email in emails: + if email.get("value"): + return str(email["value"]) + return None + + +class ScimScopeAsset(BaseAsset): + model_config = ConfigDict(extra="allow", populate_by_name=True) + dlt_config: ClassVar[DltConfig] = {"return_validated_models": True} + + enterprise_node_id: str | None = None + enterprise_slug: str | None = None + org_login: str | None = None + org_node_id: str | None = None + + @property + def scope_node_id(self) -> str: + if self.enterprise_node_id: + return self.enterprise_node_id + if self.org_node_id: + return self.org_node_id + if self.org_login: + resolved = self._lookup.org_id_for_login(self.org_login) + if resolved: + return resolved + raise ValueError("SCIM asset has no resolvable GitHub scope node ID") + + @property + def scope_name(self) -> str: + return self.enterprise_slug or self.org_login or self.scope_node_id + + @property + def scim_org_id(self) -> str: + return scim_organization_id(self.scope_node_id) + + +@app.asset( + node=NodeDef( + kind=nk.SCIM_ORGANIZATION, + description="SCIM organization exposed by a GitHub scope", + icon="building-circle-arrow-right", + color="#3B82F6", + properties=ScimNodeProperties, + ) +) +class ScimOrganization(ScimScopeAsset): + @property + def as_node(self) -> ScimNode: + return ScimNode( + id=self.scim_org_id, + kinds=[nk.SCIM_ORGANIZATION], + properties=ScimNodeProperties( + name=self.scope_name, + displayname=self.scope_name, + environmentid=self.scope_node_id, + enterprise=self.enterprise_slug, + organization=self.org_login, + source_kind="GitHub", + ), + ) + + @property + def edges(self): + return [] + + +@app.asset( + node=NodeDef( + kind=nk.SCIM_USER, + description="User provisioned through GitHub SCIM", + icon="user-gear", + color="#10B981", + properties=ScimNodeProperties, + ), + edges=[ + EdgeDef( + start=nk.SCIM_ORGANIZATION, + end=nk.SCIM_USER, + kind=ek.SCIM_CONTAINS, + description="SCIM organization contains user", + traversable=True, + ), + EdgeDef( + start=nk.SCIM_USER, + end=nk.EXTERNAL_IDENTITY, + kind=ek.SCIM_PROVISIONED, + description="SCIM user is provisioned as a GitHub external identity", + traversable=True, + ), + EdgeDef( + start=nk.OKTA_USER, + end=nk.SCIM_USER, + kind=ek.SCIM_PROVISIONED, + description="Temporary GitHound-compatible IdP-to-SCIM user correlation", + traversable=True, + ), + ], +) +class ScimUser(ScimScopeAsset): id: str - external_id: str = Field(alias="externalId") - user_name: str = Field(alias="userName") - display_name: str | None = Field(alias="displayName", default=None) - name: Name - emails: Optional[list[dict]] = Field(default_factory=list) - groups: Optional[list[dict]] = Field(default_factory=list) - roles: Optional[list[dict]] = Field(default_factory=list) - active: bool - - # Additional - org_login: str \ No newline at end of file + external_id: str | None = Field(default=None, alias="externalId") + user_name: str | None = Field(default=None, alias="userName") + display_name: str | None = Field(default=None, alias="displayName") + name: Name | None = None + emails: list[dict[str, Any]] = Field(default_factory=list) + groups: list[dict[str, Any]] = Field(default_factory=list) + roles: list[dict[str, Any]] = Field(default_factory=list) + active: bool | None = None + meta: ScimMeta | None = None + emit_legacy_correlation: bool = False + + @property + def as_node(self) -> ScimNode: + display_name = self.display_name or self.user_name or self.id + return ScimNode( + id=self.id, + kinds=[nk.SCIM_USER], + properties=ScimNodeProperties( + name=self.id, + displayname=display_name, + environmentid=self.scope_node_id, + external_id=self.external_id, + user_name=self.user_name, + enabled=self.active, + given_name=self.name.given_name if self.name else None, + family_name=self.name.family_name if self.name else None, + mail=_primary_email(self.emails), + profile_url=self.meta.location if self.meta else None, + enterprise=self.enterprise_slug, + organization=self.org_login, + source_kind="GitHub", + ), + ) + + @property + def edges(self): + yield Edge( + kind=ek.SCIM_CONTAINS, + start=EdgePath(value=self.scim_org_id, match_by="id"), + end=EdgePath(value=self.id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) + yield Edge( + kind=ek.SCIM_PROVISIONED, + start=EdgePath(value=self.id, match_by="id"), + end=ConditionalEdgePath( + kind=nk.EXTERNAL_IDENTITY, + property_matchers=[PropertyMatch(key="guid", value=self.id)], + ), + properties=EdgeProperties(traversable=True), + ) + if self.emit_legacy_correlation and self.external_id: + yield Edge( + kind=ek.SCIM_PROVISIONED, + start=EdgePath(value=self.external_id, match_by="id"), + end=EdgePath(value=self.id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) + + +@app.asset( + node=NodeDef( + kind=nk.SCIM_GROUP, + description="Group provisioned through GitHub SCIM", + icon="users-gear", + color="#EF4444", + properties=ScimNodeProperties, + ), + edges=[ + EdgeDef( + start=nk.SCIM_ORGANIZATION, + end=nk.SCIM_GROUP, + kind=ek.SCIM_CONTAINS, + description="SCIM organization contains group", + traversable=True, + ), + EdgeDef( + start=nk.SCIM_USER, + end=nk.SCIM_GROUP, + kind=ek.SCIM_MEMBER_OF, + description="SCIM user is a member of group", + traversable=True, + ), + EdgeDef( + start="Okta_Group", + end=nk.SCIM_GROUP, + kind=ek.SCIM_PROVISIONED, + description="Temporary GitHound-compatible IdP-to-SCIM group correlation", + traversable=True, + ), + ], +) +class ScimGroup(ScimScopeAsset): + id: str + external_id: str | None = Field(default=None, alias="externalId") + display_name: str = Field(alias="displayName") + members: list[dict[str, Any]] = Field(default_factory=list) + meta: ScimMeta | None = None + emit_legacy_correlation: bool = False + + @property + def legacy_okta_tenant_domain(self) -> str | None: + provider = self._lookup.enterprise_idp_for_scope(self.scope_node_id) + if not provider: + return None + foreign_idp_type, tenant_domain = detect_foreign_idp(*provider) + return tenant_domain if foreign_idp_type == "okta" else None + + @property + def as_node(self) -> ScimNode: + return ScimNode( + id=self.id, + kinds=[nk.SCIM_GROUP], + properties=ScimNodeProperties( + name=self.display_name, + displayname=self.display_name, + environmentid=self.scope_node_id, + external_id=self.external_id, + enterprise=self.enterprise_slug, + organization=self.org_login, + source_kind="GitHub", + ), + ) + + @property + def edges(self): + yield Edge( + kind=ek.SCIM_CONTAINS, + start=EdgePath(value=self.scim_org_id, match_by="id"), + end=EdgePath(value=self.id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) + for member in self.members: + member_id = member.get("value") + if member_id: + yield Edge( + kind=ek.SCIM_MEMBER_OF, + start=EdgePath(value=str(member_id), match_by="id"), + end=EdgePath(value=self.id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) + if self.emit_legacy_correlation and self.external_id: + tenant_domain = self.legacy_okta_tenant_domain + if tenant_domain: + yield Edge( + kind=ek.SCIM_PROVISIONED, + start=ConditionalEdgePath( + kind="Okta_Group", + property_matchers=[ + PropertyMatch(key="tenant_domain", value=tenant_domain), + PropertyMatch(key="name", value=self.external_id.upper()), + ], + ), + end=EdgePath(value=self.id, match_by="id"), + properties=EdgeProperties(traversable=True), + ) + else: + self._lookup.warn_missing_legacy_scim_okta_tenant_once( + self.scope_node_id, + self.scope_name, + ) + + +ScimResource = ScimUser diff --git a/src/openhound_github/models/secret_scanning_alert.py b/src/openhound_github/models/secret_scanning_alert.py index 2b887d8..4b670d6 100644 --- a/src/openhound_github/models/secret_scanning_alert.py +++ b/src/openhound_github/models/secret_scanning_alert.py @@ -149,11 +149,13 @@ def org_node_id(self) -> str | None: @property def as_node(self) -> GHNode: aid = self.node_id + repo_name = self.repository.name if self.repository else self.org_login + label = self.secret_type or "secret alert" return GHNode( kinds=[nk.SECRET_SCANNING_ALERT], properties=GHSecretScanningAlertProperties( - name=str(self.number), - displayname=self.secret_type_display_name or str(self.number), + name=f"{repo_name}-{self.number}-{label}", + displayname=f"{label} in {repo_name}", node_id=aid, environmentid=self.org_node_id, repository_name=self.repository.name if self.repository else "", diff --git a/src/openhound_github/models/workflow.py b/src/openhound_github/models/workflow.py index 403301d..ace1a1b 100644 --- a/src/openhound_github/models/workflow.py +++ b/src/openhound_github/models/workflow.py @@ -246,6 +246,9 @@ class GHWorkflowProperties(GHNodeProperties): branch: The branch where the workflow file was found. contents: The content of the workflow file. query_repository: Query for repository. + query_jobs: Query for workflow jobs. + query_execution: Query for workflow executions. + query_references: Query for workflow references (secrets and variables). query_editors: Query for editors. environment_name: The name of the environment (GitHub organization). """ @@ -263,6 +266,9 @@ class GHWorkflowProperties(GHNodeProperties): trigger_dispatch_inputs: list[str] | None = None is_pwn_requestable: bool = False query_repository: str | None = None + query_jobs: str | None = None + query_execution: str | None = None + query_references: str | None = None query_editors: str | None = None environment_name: str | None = None @@ -278,7 +284,7 @@ class GHWorkflowProperties(GHNodeProperties): EdgeDef( start=nk.REPOSITORY, end=nk.WORKFLOW, - kind=ek.HAS_WORKFLOW, + kind=ek.CONTAINS, description="Repository contains workflow", traversable=False, ), @@ -299,7 +305,7 @@ class GHWorkflowProperties(GHNodeProperties): ], ) class Workflow(BaseAsset): - """One record from `workflows` → one GH_Workflow node + GH_HasWorkflow edge from repo.""" + """One record from `workflows` → one GH_Workflow node + GH_Contains edge from repo.""" dlt_config: ClassVar[DltConfig] = {"return_validated_models": True} @@ -456,6 +462,15 @@ def _can_pwn_request_edges(self): self.repository_node_id ) + patterns = self.pull_request_target_branches + branches = self._lookup.branches_for_repository(self.repository_node_id) + target_branch_ids = [] + for branch_id, branch_name, _protected in branches: + if not patterns or any( + fnmatch.fnmatchcase(branch_name, pattern) for pattern in patterns + ): + target_branch_ids.append(branch_id) + for (role_node_id,) in read_contents: yield Edge( kind=ek.CAN_PWN_REQUEST, @@ -464,10 +479,7 @@ def _can_pwn_request_edges(self): properties=EdgeProperties(traversable=True), ) - patterns = self.pull_request_target_branches - branches = self._lookup.branches_for_repository(self.repository_node_id) - if not patterns: - for branch_id, branch_name in branches: + for branch_id in target_branch_ids: yield Edge( kind=ek.CAN_PWN_REQUEST, start=EdgePath(value=role_node_id, match_by="id"), @@ -475,18 +487,6 @@ def _can_pwn_request_edges(self): properties=EdgeProperties(traversable=True), ) - else: - for branch_id, branch_name in branches: - if any( - fnmatch.fnmatchcase(branch_name, pattern) for pattern in patterns - ): - yield Edge( - kind=ek.CAN_PWN_REQUEST, - start=EdgePath(value=role_node_id, match_by="id"), - end=EdgePath(value=branch_id, match_by="id"), - properties=EdgeProperties(traversable=True), - ) - def workflow_job_rows(self) -> list[dict[str, Any]]: document = self.document if not document: @@ -611,10 +611,13 @@ def as_node(self) -> GHNode: repository_id=self.repository_node_id, environment_name=self.org_login, environmentid=self.org_node_id, - query_repository=f"MATCH p=(:GH_Repository)-[:GH_HasWorkflow]->(:GH_Workflow {{node_id:'{wid}'}}) RETURN p", + query_repository=f"MATCH p=(:GH_Repository)-[:GH_Contains]->(:GH_Workflow {{node_id:'{wid}'}}) RETURN p", + query_jobs=f"MATCH p=(:GH_Workflow {{node_id:'{wid}'}})-[:GH_Contains]->(:GH_WorkflowJob) RETURN p", + query_execution=f"MATCH p=(:GH_Workflow {{node_id:'{wid}'}})-[:GH_Contains]->(:GH_WorkflowJob)-[:GH_Contains]->(:GH_WorkflowStep) RETURN p", + query_references=f"MATCH p=(:GH_Workflow {{node_id:'{wid}'}})-[:GH_Contains]->(:GH_WorkflowJob)-[:GH_Contains]->(step:GH_WorkflowStep) OPTIONAL MATCH p1=(step)-[:GH_UsesSecret]->() OPTIONAL MATCH p2=(step)-[:GH_UsesVariable]->() RETURN p,p1,p2", query_editors=( f"MATCH p=(role:GH_Role)-[:GH_HasRole|GH_HasBaseRole|GH_MemberOf|GH_WriteRepoContents|GH_WriteRepoPullRequests*1..]->" - f"(:GH_Repository)-[:GH_HasWorkflow]->(:GH_Workflow {{node_id:'{wid}'}}) " + f"(:GH_Repository)-[:GH_Contains]->(:GH_Workflow {{node_id:'{wid}'}}) " f"MATCH p1=(role)<-[:GH_HasRole]-(:GH_User) RETURN p,p1" ), ), @@ -623,7 +626,7 @@ def as_node(self) -> GHNode: @property def _has_workflow_edge(self): yield Edge( - kind=ek.HAS_WORKFLOW, + kind=ek.CONTAINS, start=EdgePath(value=self.repository_node_id, match_by="id"), end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), diff --git a/src/openhound_github/models/workflow_job.py b/src/openhound_github/models/workflow_job.py index 9dad1f4..c986074 100644 --- a/src/openhound_github/models/workflow_job.py +++ b/src/openhound_github/models/workflow_job.py @@ -47,6 +47,9 @@ class GHWorkflowJobProperties(GHNodeProperties): repository_name: The containing repository name. repository_id: The containing repository node ID. environment_name: The name of the GitHub organization. + query_repository: Query for repository. + query_steps: Query for workflow steps. + query_references: Query for workflow references (secrets and variables). """ job_key: str | None = None @@ -60,6 +63,9 @@ class GHWorkflowJobProperties(GHNodeProperties): repository_name: str | None = None repository_id: str | None = None environment_name: str | None = None + query_repository: str | None = None + query_steps: str | None = None + query_references: str | None = None @app.asset( @@ -73,7 +79,7 @@ class GHWorkflowJobProperties(GHNodeProperties): EdgeDef( start=nk.WORKFLOW, end=nk.WORKFLOW_JOB, - kind=ek.HAS_JOB, + kind=ek.CONTAINS, description="Workflow contains job", traversable=False, ), @@ -220,6 +226,7 @@ def is_self_hosted(self) -> bool: @property def as_node(self) -> GHNode: + jid = self.node_id return GHNode( kinds=[nk.WORKFLOW_JOB], properties=GHWorkflowJobProperties( @@ -238,6 +245,9 @@ def as_node(self) -> GHNode: repository_id=self.repository_node_id, environment_name=self.org_login, environmentid=self.org_node_id, + query_repository=f"MATCH p=(repo:GH_Repository)-[:GH_Contains]->(:GH_Workflow)-[:GH_Contains]->(:GH_WorkflowJob {{node_id:'{jid}'}}) RETURN p", + query_steps=f"MATCH p=(:GH_WorkflowJob {{node_id:'{jid}'}})-[:GH_Contains]->(:GH_WorkflowStep) RETURN p", + query_references=f"MATCH p=(:GH_WorkflowJob {{node_id:'{jid}'}})-[:GH_Contains]->(step:GH_WorkflowStep) OPTIONAL MATCH p1=(step)-[:GH_UsesSecret]->() OPTIONAL MATCH p2=(step)-[:GH_UsesVariable]->() RETURN p,p1,p2", ), ) @@ -361,7 +371,7 @@ def _uses_variable_edges(self): @property def _has_job_edge(self): yield Edge( - kind=ek.HAS_JOB, + kind=ek.CONTAINS, start=EdgePath(value=self.workflow_node_id, match_by="id"), end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), diff --git a/src/openhound_github/models/workflow_step.py b/src/openhound_github/models/workflow_step.py index 345249d..d6f643d 100644 --- a/src/openhound_github/models/workflow_step.py +++ b/src/openhound_github/models/workflow_step.py @@ -49,6 +49,8 @@ class GHWorkflowStepProperties(GHNodeProperties): repository_name: The containing repository name. repository_id: The containing repository node ID. environment_name: The name of the GitHub organization. + query_repository: Query for containing repository. + query_references: Query for secrets and variables referenced by the step. """ step_index: int | None = None @@ -66,6 +68,8 @@ class GHWorkflowStepProperties(GHNodeProperties): repository_name: str | None = None repository_id: str | None = None environment_name: str | None = None + query_repository: str | None = None + query_references: str | None = None @app.asset( @@ -79,7 +83,7 @@ class GHWorkflowStepProperties(GHNodeProperties): EdgeDef( start=nk.WORKFLOW_JOB, end=nk.WORKFLOW_STEP, - kind=ek.HAS_STEP, + kind=ek.CONTAINS, description="Workflow job contains step", traversable=False, ), @@ -97,6 +101,13 @@ class GHWorkflowStepProperties(GHNodeProperties): description="Workflow step references organization secret", traversable=False, ), + EdgeDef( + start=nk.WORKFLOW_STEP, + end=nk.ENVIRONMENT_SECRET, + kind=ek.USES_SECRET, + description="Workflow step references environment secret", + traversable=False, + ), EdgeDef( start=nk.WORKFLOW_STEP, end=nk.REPO_VARIABLE, @@ -111,20 +122,13 @@ class GHWorkflowStepProperties(GHNodeProperties): description="Workflow step references organization variable", traversable=False, ), - # EdgeDef( - # start=nk.WORKFLOW_STEP, - # end=nk.ENVIRONMENT_SECRET, - # kind=ek.USES_SECRET, - # description="Workflow step references environment secret", - # traversable=False, - # ), - # EdgeDef( - # start=nk.WORKFLOW_STEP, - # end=nk.ENVIRONMENT_VARIABLE, - # kind=ek.USES_VARIABLE, - # description="Workflow step references environment variable", - # traversable=False, - # ), + EdgeDef( + start=nk.WORKFLOW_STEP, + end=nk.ENVIRONMENT_VARIABLE, + kind=ek.USES_VARIABLE, + description="Workflow step references environment variable", + traversable=False, + ), ], ) class WorkflowStep(BaseAsset): @@ -161,6 +165,7 @@ def org_node_id(self) -> str | None: @property def as_node(self) -> GHNode: name = self.name or f"step-{self.step_index}" + sid = self.node_id return GHNode( kinds=[nk.WORKFLOW_STEP], properties=GHWorkflowStepProperties( @@ -183,6 +188,8 @@ def as_node(self) -> GHNode: repository_id=self.repository_node_id, environment_name=self.org_login, environmentid=self.org_node_id, + query_repository=f"MATCH p=(repo:GH_Repository)-[:GH_Contains]->(:GH_Workflow)-[:GH_Contains]->(:GH_WorkflowJob)-[:GH_Contains]->(:GH_WorkflowStep {{node_id:'{sid}'}}) RETURN p", + query_references=f"MATCH p=(:GH_WorkflowStep {{node_id: '{sid}'}})-[:GH_UsesSecret|GH_UsesVariable]->() RETURN p", ), ) @@ -220,6 +227,29 @@ def _uses_secret_edges(self): properties=EdgeProperties(traversable=False), ) + if self.job_environment and "${{" not in self.job_environment: + if self._lookup.environment_secret_for_environment( + ref.name, self.repository_node_id, self.job_environment + ): + yield Edge( + kind=ek.USES_SECRET, + start=EdgePath(value=self.node_id, match_by="id"), + end=ConditionalEdgePath( + kind=nk.ENVIRONMENT_SECRET, + property_matchers=[ + PropertyMatch(key="name", value=ref.name.upper()), + PropertyMatch( + key="deployment_environment_name", + value=self.job_environment, + ), + PropertyMatch( + key="repository_id", value=self.repository_node_id + ), + ], + ), + properties=EdgeProperties(traversable=False), + ) + @property def _uses_variable_edges(self): for ref in self.variable_references: @@ -254,10 +284,33 @@ def _uses_variable_edges(self): properties=EdgeProperties(traversable=False), ) + if self.job_environment and "${{" not in self.job_environment: + if self._lookup.environment_variable_for_environment( + ref.name, self.repository_node_id, self.job_environment + ): + yield Edge( + kind=ek.USES_VARIABLE, + start=EdgePath(value=self.node_id, match_by="id"), + end=ConditionalEdgePath( + kind=nk.ENVIRONMENT_VARIABLE, + property_matchers=[ + PropertyMatch(key="name", value=ref.name.upper()), + PropertyMatch( + key="deployment_environment_name", + value=self.job_environment, + ), + PropertyMatch( + key="repository_id", value=self.repository_node_id + ), + ], + ), + properties=EdgeProperties(traversable=False), + ) + @property def _has_step_edge(self): yield Edge( - kind=ek.HAS_STEP, + kind=ek.CONTAINS, start=EdgePath(value=self.job_node_id, match_by="id"), end=EdgePath(value=self.node_id, match_by="id"), properties=EdgeProperties(traversable=False), diff --git a/src/openhound_github/resources/enterprise.py b/src/openhound_github/resources/enterprise.py index a5d114e..edec491 100644 --- a/src/openhound_github/resources/enterprise.py +++ b/src/openhound_github/resources/enterprise.py @@ -2,10 +2,12 @@ from dataclasses import dataclass from dlt.sources.helpers.rest_client.client import RESTClient +from dlt.sources.helpers.rest_client.paginators import OffsetPaginator from openhound_github.graphql import ( ENTERPRISE_ADMINS_QUERY, ENTERPRISE_MEMBERS_QUERY, + ENTERPRISE_SAML_PROVIDER_QUERY, ENTERPRISE_QUERY, ENTERPRISE_SAML_QUERY, ) @@ -15,18 +17,28 @@ BaseUser, Enterprise, EnterpriseAdmin, - EnterpriseExternalIdentity, EnterpriseManagedUser, EnterpriseOrganization, EnterpriseRole, EnterpriseRoleTeam, EnterpriseRoleUser, - EnterpriseSamlProvider, EnterpriseTeam, EnterpriseTeamMember, EnterpriseTeamOrganization, EnterpriseTeamRole, EnterpriseUser, + SamlProvider, + SamlServiceProvider, + SamlAssertionConsumerService, + SamlIssuer, + ExternalIdentity, + ScimGroup, + ScimOrganization, + ScimUser, +) +from openhound_github.models.saml_helpers import ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + DEFAULT_GITHUB_WEB_ORIGIN, ) logger = logging.getLogger(__name__) @@ -37,35 +49,51 @@ class SourceContext: """Shared context for GitHub API access.""" client: RESTClient + sso_client: RESTClient | None = None + scim_client: RESTClient | None = None org_name: str | None = None enterprise_name: str | None = None + collect_enterprise_scim: bool = False + emit_legacy_scim_correlations: bool = False + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN + + +def iter_enterprise_scim_resources( + client: RESTClient, + enterprise_slug: str, + resource_kind: str, +): + if resource_kind not in {"Users", "Groups"}: + raise ValueError(f"Unsupported enterprise SCIM resource: {resource_kind}") + paginator = OffsetPaginator( + offset_param="startIndex", + limit_param="count", + limit=100, + offset=1, + total_path="totalResults", + ) + for page in client.paginate( + f"/scim/v2/enterprises/{enterprise_slug}/{resource_kind}", + params={"startIndex": 1, "count": 100}, + paginator=paginator, + data_selector="Resources", + ): + yield from page @app.resource(name="enterprise", columns=Enterprise, parallelized=True) def enterprise(ctx: SourceContext): - paginator = GraphQLCursorPaginator( - page_info_path="data.enterprise.organizations.pageInfo", - cursor_variable="after", - cursor_field="endCursor", - has_next_field="hasNextPage", - ) data = { "query": ENTERPRISE_QUERY, "variables": {"slug": ctx.enterprise_name, "after": None}, } try: - for page_data in ctx.client.paginate( - "/graphql", - method="POST", - json=data, - paginator=paginator, - data_selector="data", - ): - page_enterprise = page_data[0].get("enterprise") - - if page_enterprise: - yield page_enterprise + response = ctx.client.post("/graphql", json=data).json() + page_enterprise = (response.get("data") or {}).get("enterprise") + if page_enterprise: + yield page_enterprise except Exception as e: logger.error( f"Error in resource 'enterprise' processing enterprise '{ctx.enterprise_name}': {e}", @@ -78,12 +106,88 @@ def enterprise(ctx: SourceContext): name="enterprise_organizations", columns=EnterpriseOrganization, parallelized=True ) def enterprise_organizations(enterprise_data: Enterprise, ctx: SourceContext): - orgs = (enterprise_data.organizations or {}).get("nodes", []) - for org in orgs: + paginator = GraphQLCursorPaginator( + page_info_path="data.enterprise.organizations.pageInfo", + cursor_variable="after", + cursor_field="endCursor", + has_next_field="hasNextPage", + ) + data = { + "query": ENTERPRISE_QUERY, + "variables": {"slug": ctx.enterprise_name, "after": None}, + } + + for page_data in ctx.client.paginate( + "/graphql", + method="POST", + json=data, + paginator=paginator, + data_selector="data", + ): + for enterprise_object in page_data: + es_data = enterprise_object.get("enterprise", {}) + orgs = (es_data.get("organizations") or {}).get("nodes", []) + for org in orgs: + yield { + **org, + "enterprise_node_id": enterprise_data.id, + "enterprise_slug": ctx.enterprise_name, + } + + +@app.transformer( + name="enterprise_scim_organizations", + columns=ScimOrganization, + parallelized=True, +) +def enterprise_scim_organizations(enterprise_data: Enterprise, ctx: SourceContext): + yield { + "enterprise_node_id": enterprise_data.id, + "enterprise_slug": ctx.enterprise_name, + } + + +@app.transformer( + name="enterprise_scim_users", + columns=ScimUser, + parallelized=True, +) +def enterprise_scim_users(enterprise_data: Enterprise, ctx: SourceContext): + scim_client = ctx.scim_client or ctx.client + if not scim_client or not ctx.enterprise_name: + raise ValueError("Enterprise SCIM collection requires a client and enterprise slug") + for user in iter_enterprise_scim_resources( + scim_client, + ctx.enterprise_name, + "Users", + ): yield { - **org, + **user, "enterprise_node_id": enterprise_data.id, "enterprise_slug": ctx.enterprise_name, + "emit_legacy_correlation": ctx.emit_legacy_scim_correlations, + } + + +@app.transformer( + name="enterprise_scim_groups", + columns=ScimGroup, + parallelized=True, +) +def enterprise_scim_groups(enterprise_data: Enterprise, ctx: SourceContext): + scim_client = ctx.scim_client or ctx.client + if not scim_client or not ctx.enterprise_name: + raise ValueError("Enterprise SCIM collection requires a client and enterprise slug") + for group in iter_enterprise_scim_resources( + scim_client, + ctx.enterprise_name, + "Groups", + ): + yield { + **group, + "enterprise_node_id": enterprise_data.id, + "enterprise_slug": ctx.enterprise_name, + "emit_legacy_correlation": ctx.emit_legacy_scim_correlations, } @@ -250,12 +354,22 @@ def enterprise_roles(enterprise_data: Enterprise, ctx: SourceContext): "enterprise_slug": ctx.enterprise_name, } + yield { + "id": "members", + "name": "members", + "description": "Built-in role assigned to enterprise members", + "source": "Default", + "permissions": [], + "enterprise_node_id": enterprise_data.id, + "enterprise_slug": ctx.enterprise_name, + } + @app.transformer( name="enterprise_role_teams", columns=EnterpriseRoleTeam, parallelized=True ) def enterprise_role_teams(role: EnterpriseRole, ctx: SourceContext): - if role.id == "owners": + if role.id in {"owners", "members"}: return for page in ctx.client.paginate( @@ -276,7 +390,7 @@ def enterprise_role_teams(role: EnterpriseRole, ctx: SourceContext): name="enterprise_role_users", columns=EnterpriseRoleUser, parallelized=True ) def enterprise_role_users(role: EnterpriseRole, ctx: SourceContext): - if role.id == "owners": + if role.id in {"owners", "members"}: return for page in ctx.client.paginate( @@ -330,48 +444,119 @@ def enterprise_admins(enterprise_data: Enterprise, ctx: SourceContext): @app.transformer( - name="enterprise_saml_provider", columns=EnterpriseSamlProvider, parallelized=True + name="enterprise_saml_provider", + table_name="saml_provider", + columns=SamlProvider, + parallelized=True ) def enterprise_saml_provider(enterprise_data: Enterprise, ctx: SourceContext): - paginator = GraphQLCursorPaginator( - page_info_path="data.enterprise.ownerInfo.samlIdentityProvider.externalIdentities.pageInfo", - cursor_variable="after", - cursor_field="endCursor", - has_next_field="hasNextPage", - allow_missing_page_info=True, - ) + client = ctx.sso_client + if not client: + logger.info( + "Skipping enterprise_saml_provider for enterprise '%s': no SSO client configured", + ctx.enterprise_name, + ) + return + data = { - "query": ENTERPRISE_SAML_QUERY, - "variables": {"slug": ctx.enterprise_name, "count": 1, "after": None}, + "query": ENTERPRISE_SAML_PROVIDER_QUERY, + "variables": {"slug": ctx.enterprise_name}, } - for page_data in ctx.client.paginate( - "/graphql", - method="POST", - json=data, - paginator=paginator, - data_selector="data", - ): - for enterprise_object in page_data: - es_data = enterprise_object.get("enterprise", {}) - saml_provider = (es_data.get("ownerInfo") or {}).get("samlIdentityProvider") - if not saml_provider: - return - yield { - **{k: v for k, v in saml_provider.items() if k != "externalIdentities"}, - "enterprise_node_id": enterprise_data.id, - "enterprise_slug": ctx.enterprise_name, - } + response = client.post("/graphql", json=data).json() + enterprise_object = (response.get("data") or {}).get("enterprise", {}) + saml_provider = (enterprise_object.get("ownerInfo") or {}).get("samlIdentityProvider") + if not saml_provider: + logger.warning( + "No enterprise SAML provider returned for enterprise '%s'", + ctx.enterprise_name, + ) + return + + yield { + **saml_provider, + "environment_node_id": enterprise_data.id, + "environment_name": enterprise_data.name, + "environment_slug": enterprise_data.slug, + "environment_type": "enterprise", + "github_deployment_id": ctx.github_deployment_id, + "github_web_origin": ctx.github_web_origin, + } +@app.transformer( + name="enterprise_saml_service_provider", + table_name="saml_service_provider", + columns=SamlServiceProvider, + parallelized=True +) +def enterprise_saml_service_provider(saml_provider: SamlProvider, ctx: SourceContext): + yield { + "id": saml_provider["id"], + "issuer": saml_provider.get("issuer"), + "environment_node_id": saml_provider["environment_node_id"], + "environment_name": saml_provider["environment_name"], + "environment_slug": saml_provider["environment_slug"], + "environment_type": saml_provider["environment_type"], + "github_deployment_id": saml_provider.get("github_deployment_id"), + "github_web_origin": saml_provider.get("github_web_origin"), + } @app.transformer( - name="enterprise_external_identities", - columns=EnterpriseExternalIdentity, + name="enterprise_saml_assertion_consumer_service", + table_name="saml_assertion_consumer_service", + columns=SamlAssertionConsumerService, parallelized=True, ) -def enterprise_external_identities( - saml_provider: EnterpriseSamlProvider, ctx: SourceContext +def enterprise_saml_assertion_consumer_service( + saml_provider: SamlProvider, ctx: SourceContext ): + yield { + "environment_slug": saml_provider.get("environment_slug"), + "environment_type": saml_provider.get("environment_type"), + "environment_node_id": saml_provider.get("environment_node_id"), + "environment_name": saml_provider.get("environment_name"), + "github_deployment_id": saml_provider.get("github_deployment_id"), + "github_web_origin": saml_provider.get("github_web_origin"), + } + +@app.transformer( + name="enterprise_saml_issuer", + table_name="saml_issuer", + columns=SamlIssuer, + parallelized=True +) +def enterprise_saml_issuer(saml_provider: SamlProvider, ctx: SourceContext): + issuer = saml_provider.get("issuer") + if not issuer: + return + + yield { + "issuer": issuer, + "environment_slug": saml_provider.get("environment_slug"), + "environment_type": saml_provider.get("environment_type"), + "environment_node_id": saml_provider.get("environment_node_id"), + "environment_name": saml_provider.get("environment_name"), + "github_deployment_id": saml_provider.get("github_deployment_id"), + "github_web_origin": saml_provider.get("github_web_origin"), + } + +@app.transformer( + name="enterprise_external_identity", + table_name="external_identities", + columns=ExternalIdentity, + parallelized=True, +) +def enterprise_external_identity( + saml_provider: SamlProvider, ctx: SourceContext +): + client = ctx.sso_client + if not client: + logger.info( + "Skipping enterprise_external_identity for enterprise '%s': no SSO client configured", + ctx.enterprise_name, + ) + return + paginator = GraphQLCursorPaginator( page_info_path="data.enterprise.ownerInfo.samlIdentityProvider.externalIdentities.pageInfo", cursor_variable="after", @@ -384,7 +569,7 @@ def enterprise_external_identities( "variables": {"slug": ctx.enterprise_name, "count": 100, "after": None}, } - for page_data in ctx.client.paginate( + for page_data in client.paginate( "/graphql", method="POST", json=data, @@ -395,17 +580,20 @@ def enterprise_external_identities( es_data = enterprise_object.get("enterprise", {}) page_provider = (es_data.get("ownerInfo") or {}).get("samlIdentityProvider") if not page_provider: + logger.warning( + "No enterprise SAML provider returned while fetching external identities for enterprise '%s'", + ctx.enterprise_name, + ) return for identity in (page_provider.get("externalIdentities") or {}).get( "nodes" ) or []: yield { **identity, - "saml_provider_id": saml_provider.id, - "saml_provider_issuer": saml_provider.issuer, - "saml_provider_sso_url": saml_provider.sso_url, - "enterprise_node_id": saml_provider.enterprise_node_id, - "enterprise_slug": saml_provider.enterprise_slug, + "environment_slug": saml_provider.get("environment_slug"), + "github_deployment_id": saml_provider.get( + "github_deployment_id" + ), } @@ -415,8 +603,7 @@ def enterprise_resources(ctx: SourceContext): members_resource = enterprise_members(ctx) teams_resource = enterprise_teams(ctx) roles_resource = enterprise_roles(ctx) - saml_resource = enterprise_saml_provider(ctx) - return ( + resources = [ enterprise_resource, enterprise_resource | organizations_resource, enterprise_resource | members_resource | enterprise_users(ctx), @@ -430,6 +617,27 @@ def enterprise_resources(ctx: SourceContext): enterprise_resource | roles_resource | enterprise_role_teams(ctx), # enterprise_resource | enterprise_admin_roles(ctx), enterprise_resource | enterprise_admins(ctx), - enterprise_resource | saml_resource, - enterprise_resource | saml_resource | enterprise_external_identities(ctx), - ) + ] + + if ctx.sso_client: + saml_resource = enterprise_saml_provider(ctx) + resources.extend( + [ + enterprise_resource | saml_resource, + enterprise_resource | saml_resource | enterprise_saml_service_provider(ctx), + enterprise_resource | saml_resource | enterprise_saml_assertion_consumer_service(ctx), + enterprise_resource | saml_resource | enterprise_saml_issuer(ctx), + enterprise_resource | saml_resource | enterprise_external_identity(ctx), + ] + ) + + if ctx.collect_enterprise_scim: + resources.extend( + [ + enterprise_resource | enterprise_scim_organizations(ctx), + enterprise_resource | enterprise_scim_users(ctx), + enterprise_resource | enterprise_scim_groups(ctx), + ] + ) + + return tuple(resources) diff --git a/src/openhound_github/resources/organization.py b/src/openhound_github/resources/organization.py index 47dfa88..366d678 100644 --- a/src/openhound_github/resources/organization.py +++ b/src/openhound_github/resources/organization.py @@ -1,5 +1,3 @@ -import base64 -import binascii import logging from collections.abc import Iterable from dataclasses import dataclass, field @@ -7,7 +5,6 @@ from threading import Lock from typing import Any, Iterator -import dlt from dlt.sources.helpers import requests from dlt.sources.helpers.rest_client.client import RESTClient from dlt.sources.helpers.rest_client.paginators import ( @@ -59,6 +56,10 @@ RepoVariable, RunnerGroup, SamlProvider, + SamlServiceProvider, + SamlAssertionConsumerService, + SamlIssuer, + ScimOrganization, ScimResource, SecretScanningAlert, SelectedOrgSecret, @@ -73,6 +74,10 @@ ) from openhound_github.models.repo_role_assignment import TEAM_PERMISSION_MAP from openhound_github.models.repository_role import DEFAULT_REPO_ROLES +from openhound_github.models.saml_helpers import ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + DEFAULT_GITHUB_WEB_ORIGIN, +) logger = logging.getLogger(__name__) @@ -82,6 +87,8 @@ class OrgContext: client: RESTClient org_name: str enterprise_name: str | None = None + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN @dataclass @@ -89,6 +96,8 @@ class SourceContext: client: RESTClient organizations: list[OrgContext] = field(default_factory=list) enterprise_name: str | None = None + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN cache_lock: Lock = field(default_factory=Lock) app_cache: dict[str, dict[str, Any]] = field(default_factory=dict) actions_permissions_cache: dict[str, dict[str, Any]] = field(default_factory=dict) @@ -1707,15 +1716,18 @@ def saml_provider(ctx: SourceContext): response_data = response.get("data", {}) org_data = response_data.get("organization", {}) if response_data and org_data: - idp = org_data.get("samlIdentityProvider") - if not idp: + saml_provider = org_data.get("samlIdentityProvider") + if not saml_provider: continue yield { - **idp, - "org_node_id": org_data["id"], - "org_name": org_data["name"], - "org_login": org_name, + **saml_provider, + "environment_node_id": org_data["id"], + "environment_name": org_data["name"], + "environment_slug": org_name, + "environment_type": "org", + "github_deployment_id": org.github_deployment_id, + "github_web_origin": org.github_web_origin, } except Exception as e: logger.error( @@ -1730,7 +1742,6 @@ def external_identities(ctx: SourceContext): """Fetch external identities linked to the SAML provider. Args: - saml (SamlProvider): The SAML provider to extract identities for. ctx (SourceContext): The shared context containing the REST client and organization name. Yields: @@ -1739,6 +1750,7 @@ def external_identities(ctx: SourceContext): for org in ctx.organizations: org_name = org.org_name client = org.client + github_deployment_id = org.github_deployment_id try: paginator = GraphQLCursorPaginator( page_info_path="data.organization.samlIdentityProvider.externalIdentities.pageInfo", @@ -1767,7 +1779,11 @@ def external_identities(ctx: SourceContext): for identity in (idp.get("externalIdentities") or {}).get( "nodes" ) or []: - yield {**identity, "org_login": org_name} + yield { + **identity, + "environment_slug": org_data.get("login"), + "github_deployment_id": github_deployment_id, + } except Exception as e: logger.error( f"Error in resource 'external_identities' processing organization '{org_name}': {e}", @@ -1775,6 +1791,54 @@ def external_identities(ctx: SourceContext): ) continue +@app.transformer(name="saml_service_provider", columns=SamlServiceProvider, parallelized=True) +def saml_service_provider(saml_provider: SamlProvider, ctx: SourceContext): + """Transform SAML provider data into a normalized SAML interface representation. + + Args: + saml_provider (SamlProvider): The SAML provider to transform. + ctx (SourceContext): The shared context containing the REST client and organization name. + + Yields: + SamlInterface (SamlInterface): Normalized SAML interface record. + """ + yield { + "id": saml_provider["id"], + "issuer": saml_provider.get("issuer"), + "environment_node_id": saml_provider["environment_node_id"], + "environment_name": saml_provider["environment_name"], + "environment_slug": saml_provider["environment_slug"], + "environment_type": saml_provider["environment_type"], + "github_deployment_id": saml_provider.get("github_deployment_id"), + "github_web_origin": saml_provider.get("github_web_origin"), + } + +@app.transformer(name="saml_assertion_consumer_service", columns=SamlAssertionConsumerService, parallelized=True) +def saml_assertion_consumer_service(saml_provider: SamlProvider, ctx: SourceContext): + yield { + "environment_slug": saml_provider.get("environment_slug"), + "environment_type": saml_provider.get("environment_type"), + "environment_node_id": saml_provider.get("environment_node_id"), + "environment_name": saml_provider.get("environment_name"), + "github_deployment_id": saml_provider.get("github_deployment_id"), + "github_web_origin": saml_provider.get("github_web_origin"), + } + +@app.transformer(name="saml_issuer", columns=SamlIssuer, parallelized=True) +def saml_issuer(saml_provider: SamlProvider, ctx: SourceContext): + issuer = saml_provider.get("issuer") + if not issuer: + return + + yield { + "issuer": issuer, + "environment_slug": saml_provider.get("environment_slug"), + "environment_type": saml_provider.get("environment_type"), + "environment_node_id": saml_provider.get("environment_node_id"), + "environment_name": saml_provider.get("environment_name"), + "github_deployment_id": saml_provider.get("github_deployment_id"), + "github_web_origin": saml_provider.get("github_web_origin"), + } @app.resource(name="scim_users", columns=ScimResource, parallelized=True) def scim_users(ctx: SourceContext): @@ -1815,6 +1879,18 @@ def scim_users(ctx: SourceContext): continue +@app.transformer( + name="org_scim_organizations", + columns=ScimOrganization, + parallelized=True, +) +def org_scim_organizations(org: Organization): + yield { + "org_login": org.login, + "org_node_id": org.node_id, + } + + def organization_resources(ctx: SourceContext): org_resource = organizations(ctx) roles_resource = org_roles(ctx) @@ -1834,6 +1910,7 @@ def organization_resources(ctx: SourceContext): organization_secrets_resource = organization_secrets(ctx) organization_vars_resource = organization_variables(ctx) projected_enterprise_teams_resource = projected_enterprise_teams(ctx) + saml_resource = saml_provider(ctx) return ( org_resource, @@ -1852,6 +1929,7 @@ def organization_resources(ctx: SourceContext): repos_resource | repository_variables(ctx), teams_resource, projected_enterprise_teams_resource, + org_resource | org_scim_organizations(), teams_resource | team_members(ctx), teams_resource | team_roles(), teams_resource | team_repo_role_assignments(ctx, repo_roles_base), @@ -1860,7 +1938,10 @@ def organization_resources(ctx: SourceContext): repositories_graphql_resource | branches(ctx), branch_prot_rules_resource, secret_scanning_alerts(ctx), - saml_provider(ctx), + saml_resource, + saml_resource | saml_service_provider(ctx), + saml_resource | saml_assertion_consumer_service(ctx), + saml_resource | saml_issuer(ctx), external_identities(ctx), workflows_resource, workflows_resource | workflow_jobs(), diff --git a/src/openhound_github/source.py b/src/openhound_github/source.py index 5da35cb..f6e6989 100644 --- a/src/openhound_github/source.py +++ b/src/openhound_github/source.py @@ -1,13 +1,13 @@ import logging -import time from dataclasses import dataclass, field from threading import Lock -from typing import Any, Optional, Union +from typing import Any, Union import dlt from dlt.common.configuration import configspec from dlt.common.configuration.specs import CredentialsConfiguration from dlt.sources.helpers import requests +from dlt.sources.helpers.rest_client.auth import AuthConfigBase from dlt.sources.helpers.rest_client.auth import BearerTokenAuth from dlt.sources.helpers.rest_client.client import RESTClient from dlt.sources.helpers.rest_client.paginators import ( @@ -21,6 +21,11 @@ ) from openhound_github.helpers import github_retry_policy from openhound_github.main import app +from openhound_github.models.saml_helpers import ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + DEFAULT_GITHUB_WEB_ORIGIN, + github_deployment_context, +) from .resources.enterprise import enterprise_resources from .resources.organization import organization_resources @@ -33,13 +38,21 @@ class OrgContext: client: RESTClient org_name: str enterprise_name: str | None = None + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN @dataclass class SourceContext: organizations: list[OrgContext] | None = field(default_factory=list) client: RESTClient | None = None + sso_client: RESTClient | None = None + scim_client: RESTClient | None = None enterprise_name: str | None = None + collect_enterprise_scim: bool = False + emit_legacy_scim_correlations: bool = False + github_deployment_id: str = DEFAULT_GITHUB_DEPLOYMENT_ID + github_web_origin: str = DEFAULT_GITHUB_WEB_ORIGIN cache_lock: Lock = field(default_factory=Lock) app_cache: dict[str, dict[str, Any]] = field(default_factory=dict) actions_permissions_cache: dict[str, dict[str, Any]] = field(default_factory=dict) @@ -66,6 +79,8 @@ class GithubEnterpriseAppCredentials(CredentialsConfiguration): app_id: str = None key_path: str = None enterprise_name: str = None + pat_token: str | None = None + scim_token: str | None = None api_uri: str = "https://api.github.com" @property @@ -89,6 +104,7 @@ def auth(self) -> str: @configspec class GithubTokenCredentials(GithubCredentials): token: str = None + scim_token: str | None = None @property def auth(self) -> str: @@ -105,6 +121,8 @@ def source( GithubEnterpriseAppCredentials, GithubOrgAppCredentials, GithubTokenCredentials ] = dlt.secrets.value, host: str = "https://api.github.com", + collect_enterprise_scim: bool | None = dlt.config.value, + emit_legacy_scim_correlations: bool | None = dlt.config.value, ): """DLT source, defines GitHub collection resources and transformers. @@ -112,8 +130,9 @@ def source( credentials (Union[GithubEnterpriseAppCredentials, GithubOrgAppCredentials, GithubTokenCredentials]): The GitHub credentials. host (str): The base GitHub API URL used for API calls. """ + github_deployment_id, github_web_origin = github_deployment_context(host) - def client(auth: GitHubAppInstallationAuth) -> RESTClient: + def client(auth: AuthConfigBase) -> RESTClient: return RESTClient( base_url=host, headers={ @@ -128,8 +147,23 @@ def client(auth: GitHubAppInstallationAuth) -> RESTClient: ).session, ) + def token_client(token: str) -> RESTClient: + return client(BearerTokenAuth(token=token)) + if credentials.auth == "enterprise_app": - ctx = SourceContext(enterprise_name=credentials.enterprise_name) + ctx = SourceContext( + enterprise_name=credentials.enterprise_name, + collect_enterprise_scim=bool(collect_enterprise_scim), + emit_legacy_scim_correlations=bool(emit_legacy_scim_correlations), + github_deployment_id=github_deployment_id, + github_web_origin=github_web_origin, + ) + if credentials.pat_token: + ctx.sso_client = token_client(credentials.pat_token) + if credentials.scim_token: + ctx.scim_client = token_client(credentials.scim_token) + elif credentials.pat_token: + ctx.scim_client = ctx.sso_client github_app_session = GithubApp( client_id=credentials.client_id, private_key_path=credentials.key_path, @@ -148,6 +182,8 @@ def client(auth: GitHubAppInstallationAuth) -> RESTClient: GitHubAppInstallationAuth(installation=org_installation) ), enterprise_name=credentials.enterprise_name, + github_deployment_id=github_deployment_id, + github_web_origin=github_web_origin, ) ) if installation.target_type == "Enterprise": @@ -163,7 +199,11 @@ def client(auth: GitHubAppInstallationAuth) -> RESTClient: return (*enterprise_resources(ctx), *organization_resources(ctx)) elif credentials.auth == "org_app": - ctx = SourceContext(enterprise_name=None) + ctx = SourceContext( + enterprise_name=None, + github_deployment_id=github_deployment_id, + github_web_origin=github_web_origin, + ) org_installation = GithubInstallation( installation_id=credentials.install_id, client_id=credentials.client_id, @@ -173,25 +213,40 @@ def client(auth: GitHubAppInstallationAuth) -> RESTClient: OrgContext( org_name=credentials.org_name, client=client(GitHubAppInstallationAuth(installation=org_installation)), + github_deployment_id=github_deployment_id, + github_web_origin=github_web_origin, ) ) return organization_resources(ctx) else: - ctx = SourceContext() + if credentials.enterprise_name: + token_api_client = token_client(credentials.token) + ctx = SourceContext( + client=token_api_client, + sso_client=token_api_client, + scim_client=token_client(credentials.scim_token) + if credentials.scim_token + else token_api_client, + enterprise_name=credentials.enterprise_name, + collect_enterprise_scim=bool(collect_enterprise_scim), + emit_legacy_scim_correlations=bool(emit_legacy_scim_correlations), + github_deployment_id=github_deployment_id, + github_web_origin=github_web_origin, + ) + return enterprise_resources(ctx) + + ctx = SourceContext( + github_deployment_id=github_deployment_id, + github_web_origin=github_web_origin, + ) ctx.organizations.append( OrgContext( org_name=credentials.org_name, - client=RESTClient( - base_url=host, - headers={ - "Accept": "application/vnd.github+json", - "X-GitHub-Api-Version": "2022-11-28", - }, - auth=BearerTokenAuth(token=credentials.token), - paginator=HeaderLinkPaginator(), - ), + client=token_client(credentials.token), + github_deployment_id=github_deployment_id, + github_web_origin=github_web_origin, ) ) return organization_resources(ctx) diff --git a/src/openhound_github/transforms.py b/src/openhound_github/transforms.py index 8edc15c..bcc5630 100644 --- a/src/openhound_github/transforms.py +++ b/src/openhound_github/transforms.py @@ -1,6 +1,94 @@ import duckdb +def ensure_optional_input_tables( + con: duckdb.DuckDBPyConnection, schema: str = "github" +) -> None: + """Create typed empty tables for zero-row branch-policy inputs. + + DLT omits resources that yield no rows. Enterprise GitHub App collection can + legitimately have no branches, branch-protection rules, or repository roles, + while the derived branch transforms still need stable input schemas. + """ + con.execute(f""" + CREATE TABLE IF NOT EXISTS {schema}.branches ( + id VARCHAR, + branch_protection_rule JSON, + repository_node_id VARCHAR + ); + CREATE TABLE IF NOT EXISTS {schema}.branch_protection_rules ( + id VARCHAR, + repository_node_id VARCHAR, + pattern VARCHAR, + requires_approving_reviews BOOLEAN, + lock_branch BOOLEAN, + restricts_pushes BOOLEAN, + is_admin_enforced BOOLEAN, + bypass_pull_request_allowances JSON, + push_allowances JSON, + blocks_creations BOOLEAN + ); + CREATE TABLE IF NOT EXISTS {schema}.repo_roles ( + id BIGINT, + repository_node_id VARCHAR, + permissions JSON + ); + CREATE TABLE IF NOT EXISTS {schema}.organization_variables ( + name VARCHAR, + org_login VARCHAR, + value VARCHAR, + visibility VARCHAR, + selected_repositories_url VARCHAR, + created_at TIMESTAMP, + updated_at TIMESTAMP + ); + CREATE TABLE IF NOT EXISTS {schema}.selected_organization_variables ( + org_login VARCHAR, + variable_name VARCHAR, + repository_node_id VARCHAR + ); + """) + con.execute(f""" + ALTER TABLE {schema}.branches + ADD COLUMN IF NOT EXISTS branch_protection_rule JSON; + + ALTER TABLE {schema}.branch_protection_rules + ADD COLUMN IF NOT EXISTS id VARCHAR; + ALTER TABLE {schema}.branch_protection_rules + ADD COLUMN IF NOT EXISTS repository_node_id VARCHAR; + ALTER TABLE {schema}.branch_protection_rules + ADD COLUMN IF NOT EXISTS pattern VARCHAR; + ALTER TABLE {schema}.branch_protection_rules + ADD COLUMN IF NOT EXISTS requires_approving_reviews BOOLEAN; + ALTER TABLE {schema}.branch_protection_rules + ADD COLUMN IF NOT EXISTS lock_branch BOOLEAN; + ALTER TABLE {schema}.branch_protection_rules + ADD COLUMN IF NOT EXISTS restricts_pushes BOOLEAN; + ALTER TABLE {schema}.branch_protection_rules + ADD COLUMN IF NOT EXISTS is_admin_enforced BOOLEAN; + ALTER TABLE {schema}.branch_protection_rules + ADD COLUMN IF NOT EXISTS bypass_pull_request_allowances JSON; + ALTER TABLE {schema}.branch_protection_rules + ADD COLUMN IF NOT EXISTS push_allowances JSON; + ALTER TABLE {schema}.branch_protection_rules + ADD COLUMN IF NOT EXISTS blocks_creations BOOLEAN; + + ALTER TABLE {schema}.repo_roles + ADD COLUMN IF NOT EXISTS permissions JSON; + + ALTER TABLE {schema}.organization_variables + ADD COLUMN IF NOT EXISTS name VARCHAR; + ALTER TABLE {schema}.organization_variables + ADD COLUMN IF NOT EXISTS org_login VARCHAR; + + ALTER TABLE {schema}.selected_organization_variables + ADD COLUMN IF NOT EXISTS org_login VARCHAR; + ALTER TABLE {schema}.selected_organization_variables + ADD COLUMN IF NOT EXISTS variable_name VARCHAR; + ALTER TABLE {schema}.selected_organization_variables + ADD COLUMN IF NOT EXISTS repository_node_id VARCHAR; + """) + # TODO: # This can be optimized to generate the actor_branch_gates table # in one go instead of intermedaite tables @@ -93,6 +181,8 @@ def transforms(con: duckdb.DuckDBPyConnection, schema: str = "github") -> None: con: The DuckDB connection to use for creating computed tables. schema: The DuckDB schema name containing the source tables. """ + + ensure_optional_input_tables(con, schema) join_branch_bpr(con, schema) actor_allowances(con, schema) unprotected_branches(con, schema) diff --git a/tests/test_enterprise_capabilities.py b/tests/test_enterprise_capabilities.py new file mode 100644 index 0000000..f8d0692 --- /dev/null +++ b/tests/test_enterprise_capabilities.py @@ -0,0 +1,135 @@ +from types import SimpleNamespace +from unittest.mock import MagicMock + +import pytest + +from openhound_github.kinds import edges as ek +from openhound_github.models.enterprise_role import ( + ENTERPRISE_PERMISSION_EDGES, + EnterpriseRole, +) +from openhound_github.models.enterprise_user import EnterpriseUser +from openhound_github.resources.enterprise import ( + SourceContext, + enterprise_role_teams, + enterprise_role_users, + enterprise_roles, +) + + +class _FakeResponse: + def __init__(self, payload: dict): + self._payload = payload + + def json(self) -> dict: + return self._payload + + +class _FakeClient: + def __init__(self, payload: dict | None = None): + self.payload = payload or {"roles": []} + self.get_calls: list[str] = [] + self.paginate_calls: list[tuple[str, dict]] = [] + + def get(self, path: str): + self.get_calls.append(path) + return _FakeResponse(self.payload) + + def paginate(self, path: str, **kwargs): + self.paginate_calls.append((path, kwargs)) + return iter([]) + + +def _role(permissions: list[str]) -> EnterpriseRole: + role = EnterpriseRole( + id=42, + name="security-manager", + source="Organization", + permissions=permissions, + enterprise_node_id="ENT_NODE_1", + enterprise_slug="example-enterprise", + ) + lookup = MagicMock() + lookup.enterprise_id.return_value = "ENT_NODE_1" + role._lookup = lookup + return role + + +@pytest.mark.parametrize( + ("permission", "edge_kind", "traversable"), + [ + ("manage_enterprise_admins", ek.MANAGE_ENTERPRISE_ADMINS, True), + ("manage_enterprise_members", ek.MANAGE_ENTERPRISE_MEMBERS, True), + ( + "manage_enterprise_organization_admins", + ek.MANAGE_ENTERPRISE_ORGANIZATION_ADMINS, + True, + ), + ("write_enterprise_sso", ek.WRITE_ENTERPRISE_SSO, False), + ( + "create_enterprise_organizations", + ek.CREATE_ENTERPRISE_ORGANIZATIONS, + False, + ), + ], +) +def test_enterprise_permissions_become_capability_edges( + permission: str, + edge_kind: str, + traversable: bool, +) -> None: + edges = list(_role([permission]).edges) + + capability = edges[1] + assert capability.kind == edge_kind + assert capability.end.value == "ENT_NODE_1" + assert capability.properties.traversable is traversable + + +def test_every_enterprise_permission_has_a_unique_edge_kind() -> None: + assert len(ENTERPRISE_PERMISSION_EDGES) == 23 + assert len({kind for kind, _ in ENTERPRISE_PERMISSION_EDGES.values()}) == 23 + + +def test_unknown_enterprise_permission_is_preserved_but_not_invented_as_edge() -> None: + edges = list(_role(["future_permission_not_yet_modeled"]).edges) + + assert [edge.kind for edge in edges] == [ek.CONTAINS] + + +def test_every_enterprise_user_receives_builtin_members_role() -> None: + user = EnterpriseUser( + id="USER_NODE_1", + login="alice", + enterprise_node_id="ENT_NODE_1", + enterprise_slug="example-enterprise", + has_direct_enterprise_membership=False, + ) + lookup = MagicMock() + lookup.enterprise_id.return_value = "ENT_NODE_1" + user._lookup = lookup + + edges = list(user.edges) + + assert [edge.kind for edge in edges] == [ek.HAS_ROLE] + assert edges[0].end.value.endswith("_members") + + +def test_enterprise_roles_include_synthetic_members_role() -> None: + client = _FakeClient({"roles": []}) + ctx = SourceContext(client=client, enterprise_name="example-enterprise") + + rows = list(enterprise_roles.__wrapped__(SimpleNamespace(id="ENT_NODE_1"), ctx)) + + assert [row["id"] for row in rows] == ["owners", "members"] + + +@pytest.mark.parametrize("transformer", [enterprise_role_users, enterprise_role_teams]) +def test_synthetic_enterprise_roles_do_not_call_assignment_endpoints(transformer) -> None: + client = _FakeClient() + ctx = SourceContext(client=client, enterprise_name="example-enterprise") + role = _role([]) + role.id = "members" + + assert list(transformer.__wrapped__(role, ctx)) == [] + assert client.paginate_calls == [] diff --git a/tests/test_enterprise_resources.py b/tests/test_enterprise_resources.py new file mode 100644 index 0000000..25cb818 --- /dev/null +++ b/tests/test_enterprise_resources.py @@ -0,0 +1,93 @@ +from types import SimpleNamespace + +from openhound_github.resources.enterprise import ( + SourceContext, + enterprise, + enterprise_organizations, +) + + +class _FakeResponse: + def __init__(self, payload: dict): + self._payload = payload + + def json(self) -> dict: + return self._payload + + +class _FakeClient: + def __init__(self, payload: dict, pages: list[dict] | None = None): + self.payload = payload + self.pages = pages or [] + self.post_calls: list[tuple[str, dict]] = [] + self.paginate_calls: list[tuple[str, dict]] = [] + + def post(self, path: str, json: dict): + self.post_calls.append((path, json)) + return _FakeResponse(self.payload) + + def paginate(self, path: str, **kwargs): + self.paginate_calls.append((path, kwargs)) + return iter(self.pages) + + +def test_enterprise_resource_yields_single_record() -> None: + client = _FakeClient( + { + "data": { + "enterprise": { + "id": "E_1", + "slug": "acme", + "organizations": { + "nodes": [{"id": "O_1", "login": "org-1"}], + "pageInfo": {"hasNextPage": True, "endCursor": "cursor-1"}, + }, + } + } + } + ) + ctx = SourceContext(client=client, enterprise_name="acme") + + rows = list(enterprise(ctx)) + + assert len(rows) == 1 + assert rows[0].id == "E_1" + assert len(client.post_calls) == 1 + + +def test_enterprise_organizations_paginates_all_pages() -> None: + client = _FakeClient( + payload={}, + pages=[ + [ + { + "enterprise": { + "organizations": { + "nodes": [{"id": "O_1", "login": "org-1"}], + "pageInfo": {"hasNextPage": True, "endCursor": "cursor-1"}, + } + } + } + ], + [ + { + "enterprise": { + "organizations": { + "nodes": [{"id": "O_2", "login": "org-2"}], + "pageInfo": {"hasNextPage": False, "endCursor": None}, + } + } + } + ], + ], + ) + ctx = SourceContext(client=client, enterprise_name="acme") + enterprise_data = SimpleNamespace(id="E_1") + + rows = list(enterprise_organizations.__wrapped__(enterprise_data, ctx)) + + assert rows == [ + {"id": "O_1", "login": "org-1", "enterprise_node_id": "E_1", "enterprise_slug": "acme"}, + {"id": "O_2", "login": "org-2", "enterprise_node_id": "E_1", "enterprise_slug": "acme"}, + ] + assert len(client.paginate_calls) == 1 diff --git a/tests/test_environment_model.py b/tests/test_environment_model.py new file mode 100644 index 0000000..975ee53 --- /dev/null +++ b/tests/test_environment_model.py @@ -0,0 +1,164 @@ +from unittest.mock import MagicMock + +from openhound_github.kinds import edges as ek +from openhound_github.models.environment import DeploymentBranchPolicy +from openhound_github.models.environment import Environment +from openhound_github.models.environment_branch_policy import EnvironmentBranchPolicy + +def _make_environment() -> Environment: + env = Environment( + id=161088068, + node_id="MDExOkVudmlyb25tZW50MTYxMDg4MDY4", + name="staging", + url="https://api.github.com/repos/github/hello-world/environments/staging", + html_url="https://github.com/github/hello-world/deployments/activity_log?environments_filter=staging", + created_at="2020-11-23T22:00:40Z", + updated_at="2020-11-23T22:00:40Z", + protection_rules=[ + { + "id": 3736, + "node_id": "MDQ6R2F0ZTM3MzY=", + "type": "wait_timer", + "wait_timer": 30, + }, + { + "id": 3755, + "node_id": "MDQ6R2F0ZTM3NTU=", + "prevent_self_review": False, + "type": "required_reviewers", + "reviewers": [ + { + "type": "User", + "reviewer": { + "login": "octocat", + "id": 1, + "node_id": "MDQ6VXNlcjE=", + "avatar_url": "", + "gravatar_id": "", + "url": "", + "html_url": "", + "followers_url": "", + "following_url": "", + "gists_url": "", + "starred_url": "", + "subscriptions_url": "", + "organizations_url": "", + "repos_url": "", + "events_url": "", + "received_events_url": "", + "type": "User", + "site_admin": False, + }, + }, + { + "type": "Team", + "reviewer": { + "id": 1, + "node_id": "MDQ6VGVhbTE=", + "url": "", + "html_url": "", + "name": "Justice League", + "slug": "justice-league", + "description": "A great team.", + "privacy": "closed", + "notification_setting": "notifications_enabled", + "permission": "admin", + "members_url": "", + "repositories_url": "", + "parent": None, + }, + }, + ], + }, + { + "id": 3756, + "node_id": "MDQ6R2F0ZTM3NTY=", + "type": "branch_policy", + }, + ], + deployment_branch_policy={ + "protected_branches": False, + "custom_branch_policies": True, + }, + org_login="github", + repository_name="hello-world", + repository_full_name="github/hello-world", + repository_node_id="R_123", + ) + lookup = MagicMock() + lookup.org_id_for_login.return_value = "O_123" + env._lookup = lookup + return env + +def _make_environment_branch_policy(name: str) -> EnvironmentBranchPolicy: + return EnvironmentBranchPolicy( + id=1, + node_id="POLICY_1", + name=name, + environment_node_id="ENV_1", + environment_name="production", + repository_name="hello-world", + repository_node_id="R_123", + org_login="github", + ) + +def test_environment_node_surfaces_protection_rule_properties() -> None: + env = _make_environment() + + node = env.as_node + + assert node.properties.wait_timer == 30 + assert node.properties.prevent_self_review is False + assert node.properties.reviewer_count == 2 + assert node.properties.custom_branch_policies is True + assert node.properties.protected_branches is False + + +def test_environment_edges_include_required_reviewer_relationships() -> None: + env = _make_environment() + + edges = list(env.edges) + + reviewer_edges = [edge for edge in edges if edge.kind == ek.APPROVES_DEPLOYMENT_TO] + assert len(reviewer_edges) == 2 + assert {edge.start.value for edge in reviewer_edges} == { + "MDQ6VXNlcjE=", + "MDQ6VGVhbTE=", + } + +def test_environment_branch_policy_single_star_does_not_cross_slashes() -> None: + policy = _make_environment_branch_policy("release/*") + + assert policy.matches_branch("release/v1") is True + assert policy.matches_branch("release/v1/hotfix") is False + assert policy.matches_branch("foo/bar") is False + + +def test_environment_branch_policy_double_star_crosses_slashes() -> None: + policy = _make_environment_branch_policy("release/**/*") + + assert policy.matches_branch("release/v1/hotfix") is True + assert policy.matches_branch("release/v1") is True + assert policy.matches_branch("release/") is False + +def test_protected_branches_only_without_any_bpr_allows_all_branches() -> None: + env = _make_environment() + env.deployment_branch_policy = DeploymentBranchPolicy( + protected_branches=True, + custom_branch_policies=False, + ) + env._lookup.branches_with_bpr.return_value = [] + env._lookup.branches_for_repository.return_value = [ + ("B_main", "main", False), + ("B_release", "release/v1", False), + ] + + deploy_edges = [ + edge for edge in env.edges if edge.kind == ek.CAN_DEPLOY_TO_ENVIRONMENT + ] + + assert {edge.start.value for edge in deploy_edges} == { + "R_123", + "B_main", + "B_release", + } diff --git a/tests/test_external_identity_tenant_matching.py b/tests/test_external_identity_tenant_matching.py new file mode 100644 index 0000000..a28593e --- /dev/null +++ b/tests/test_external_identity_tenant_matching.py @@ -0,0 +1,218 @@ +from dataclasses import asdict + +import pytest + +from openhound_github.kinds import edges as ek +from openhound_github.models.external_identity import ExternalIdentity + + +class _Lookup: + def __init__( + self, + issuer: str, + sso_url: str | None, + environment_type: str = "org", + ): + self.issuer = issuer + self.sso_url = sso_url + self.environment_type = environment_type + + def idp_for_environment(self, slug: str): + return [ + ( + f"idp:{slug}", + self.issuer, + self.sso_url, + f"environment:{slug}", + slug, + self.environment_type, + ) + ] + + +def _identity( + slug: str, + issuer: str, + sso_url: str | None, + saml_identity: dict, + environment_type: str = "org", +) -> ExternalIdentity: + identity = ExternalIdentity.model_validate( + { + "guid": f"guid:{slug}", + "id": f"external-identity:{slug}", + "samlIdentity": saml_identity, + "scimIdentity": None, + "user": {"id": f"github-user:{slug}", "login": "duplicate"}, + "environment_slug": slug, + } + ) + identity._lookup = _Lookup(issuer, sso_url, environment_type) + return identity + + +def _foreign_user_edges(identity: ExternalIdentity): + edges = list(identity.edges) + maps_to = next( + edge + for edge in edges + if edge.kind == ek.MAPS_TO_USER + and getattr(edge.end, "property_matchers", None) + ) + synced_to = next(edge for edge in edges if edge.kind == ek.SYNCED_TO_GH_USER) + return maps_to, synced_to + + +def test_external_identity_scopes_duplicate_okta_username_by_tenant() -> None: + first = _foreign_user_edges( + _identity( + "first-org", + "http://www.okta.com/example", + "https://first.example.okta.com/app/github/sso/saml", + {"username": "duplicate@example.com"}, + ) + ) + second = _foreign_user_edges( + _identity( + "second-org", + "http://www.okta.com/example", + "https://second.example.okta.com/app/github/sso/saml", + {"username": "duplicate@example.com"}, + environment_type="enterprise", + ) + ) + + for edges, tenant_domain in ( + (first, "first.example.okta.com"), + (second, "second.example.okta.com"), + ): + for endpoint in (edges[0].end, edges[1].start): + assert endpoint.kind == "Okta_User" + assert { + matcher.key: matcher.value for matcher in endpoint.property_matchers + } == { + "tenant_domain": tenant_domain, + "name": "DUPLICATE@EXAMPLE.COM", + } + + emitted_endpoint = asdict(first[0].end) + assert emitted_endpoint["kind"] == "Okta_User" + assert emitted_endpoint["match_by"] == "property" + assert [ + {"key": matcher["key"], "value": matcher["value"]} + for matcher in emitted_endpoint["property_matchers"] + ] == [ + {"key": "tenant_domain", "value": "first.example.okta.com"}, + {"key": "name", "value": "DUPLICATE@EXAMPLE.COM"}, + ] + + +def test_external_identity_uses_pingone_environment_scope() -> None: + maps_to, synced_to = _foreign_user_edges( + _identity( + "pingone-org", + "https://auth.pingone.com/ping-environment-id/saml20/idp/sso", + None, + {"username": "opaque-pingone-subject"}, + ) + ) + + for endpoint in (maps_to.end, synced_to.start): + assert endpoint.kind == "PingOne_User" + assert { + matcher.key: matcher.value for matcher in endpoint.property_matchers + } == { + "environmentid": "ping-environment-id", + "name": "OPAQUE-PINGONE-SUBJECT", + } + + +def _entra_identity( + tenant_claim: str | None = "11111111-2222-3333-4444-555555555555", + object_id_claim: str | None = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", +) -> ExternalIdentity: + attributes = [] + if tenant_claim is not None: + attributes.append( + { + "name": "http://schemas.microsoft.com/identity/claims/tenantid", + "value": tenant_claim, + } + ) + if object_id_claim is not None: + attributes.append( + { + "name": ( + "http://schemas.microsoft.com/identity/claims/objectidentifier" + ), + "value": object_id_claim, + } + ) + return _identity( + "entra-org", + "https://sts.windows.net/11111111-2222-3333-4444-555555555555/", + ( + "https://login.microsoftonline.com/" + "11111111-2222-3333-4444-555555555555/saml2" + ), + { + "username": "opaque-pairwise-name-id", + "attributes": attributes, + }, + ) + + +def test_entra_external_identity_uses_explicit_saml_claims() -> None: + maps_to, synced_to = _foreign_user_edges(_entra_identity()) + + for endpoint in (maps_to.end, synced_to.start): + assert endpoint.kind == "AZUser" + assert { + matcher.key: matcher.value for matcher in endpoint.property_matchers + } == { + "tenantid": "11111111-2222-3333-4444-555555555555".upper(), + "objectid": "AAAAAAAA-BBBB-CCCC-DDDD-EEEEEEEEEEEE", + } + + +@pytest.mark.parametrize( + ("tenant_claim", "object_id_claim"), + [ + (None, "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"), + ("11111111-2222-3333-4444-555555555555", None), + ( + "99999999-8888-7777-6666-555555555555", + "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", + ), + ], +) +def test_entra_external_identity_omits_unsafe_foreign_user_edges( + tenant_claim: str | None, + object_id_claim: str | None, +) -> None: + edges = list(_entra_identity(tenant_claim, object_id_claim).edges) + + assert ek.SYNCED_TO_GH_USER not in {edge.kind for edge in edges} + assert not any( + edge.kind == ek.MAPS_TO_USER + and getattr(edge.end, "property_matchers", None) + for edge in edges + ) + + +def test_external_identity_omits_okta_edges_without_tenant_scope() -> None: + edges = list( + _identity( + "unscoped-org", + "http://www.okta.com/example", + None, + {"username": "user@example.com"}, + ).edges + ) + + assert ek.SYNCED_TO_GH_USER not in {edge.kind for edge in edges} + assert not any( + edge.kind == ek.MAPS_TO_USER + and getattr(edge.end, "property_matchers", None) + for edge in edges + ) diff --git a/tests/test_org_secret_models.py b/tests/test_org_secret_models.py new file mode 100644 index 0000000..c746ae2 --- /dev/null +++ b/tests/test_org_secret_models.py @@ -0,0 +1,91 @@ +from datetime import datetime +from unittest.mock import MagicMock + +from openhound_github.kinds import edges as ek +from openhound_github.models.org_role import OrgRole +from openhound_github.models.org_secret import OrgSecret + + +def _secret( + visibility: str, + creation_flags: tuple[bool, bool, bool, bool], +) -> OrgSecret: + secret = OrgSecret( + name="DEPLOY_TOKEN", + created_at=datetime.now(), + visibility=visibility, + org_login="acme", + ) + lookup = MagicMock() + lookup.org_id_for_login.return_value = "O_1" + lookup.members_can_create_repository.return_value = creation_flags + secret._lookup = lookup + return secret + + +def test_all_visibility_secret_requires_an_actual_member_creation_capability() -> None: + secret = _secret("all", (False, False, False, False)) + + edges = list(secret._composed_read_secret_edges) + + assert [edge.start.value for edge in edges] == ["O_1_owners"] + + +def test_all_visibility_secret_allows_any_repository_creation_capability() -> None: + secret = _secret("all", (True, False, False, False)) + + edges = list(secret._composed_read_secret_edges) + + assert [edge.start.value for edge in edges] == ["O_1_owners", "O_1_members"] + assert "GH_CanCreateRepositories" in edges[1].properties.query_composition + assert "GH_CanCreatePublicRepositories" in edges[1].properties.query_composition + assert "GH_CanCreateInternalRepositories" in edges[1].properties.query_composition + assert "GH_CanCreatePrivateRepositories" in edges[1].properties.query_composition + + +def test_private_visibility_secret_only_allows_private_or_internal_creation() -> None: + secret = _secret("private", (True, True, False, False)) + + edges = list(secret._composed_read_secret_edges) + + assert [edge.start.value for edge in edges] == ["O_1_owners"] + query = edges[0].properties.query_composition + assert "GH_CanCreateInternalRepositories" in query + assert "GH_CanCreatePrivateRepositories" in query + assert "GH_CanCreateRepositories" not in query + assert "GH_CanCreatePublicRepositories" not in query + + +def test_private_visibility_secret_allows_internal_repository_creation() -> None: + secret = _secret("private", (False, False, True, False)) + + edges = list(secret._composed_read_secret_edges) + + assert [edge.start.value for edge in edges] == ["O_1_owners", "O_1_members"] + + +def test_selected_visibility_secret_does_not_emit_latent_read_edges() -> None: + secret = _secret("selected", (True, True, True, True)) + + assert list(secret._composed_read_secret_edges) == [] + + +def test_owners_always_emit_repository_creation_edges_needed_for_secret_paths() -> None: + role = OrgRole( + id=1, + name="owners", + type="default", + base_role="admin", + created_at=datetime.now(), + org_node_id="O_1", + org_login="acme", + ) + lookup = MagicMock() + lookup.members_can_create_repository.return_value = (False, False, False, False) + role._lookup = lookup + + kinds = {edge.kind for edge in role._can_create_repos_edge} + + assert ek.CAN_CREATE_REPOSITORIES in kinds + assert ek.CAN_CREATE_PUBLIC_REPOSITORIES in kinds + assert ek.CAN_CREATE_PRIVATE_REPOSITORIES in kinds diff --git a/tests/test_saml_helpers.py b/tests/test_saml_helpers.py new file mode 100644 index 0000000..4bc1f58 --- /dev/null +++ b/tests/test_saml_helpers.py @@ -0,0 +1,70 @@ +from types import SimpleNamespace + +import pytest + +from openhound_github.models.saml_helpers import ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + DEFAULT_GITHUB_WEB_ORIGIN, + ENTRA_OBJECT_ID_CLAIM, + github_deployment_context, + github_enterprise_acs_url, + github_enterprise_saml_service_provider_id, + github_org_acs_url, + github_org_saml_service_provider_id, + saml_account_match_values, + saml_attribute_match_values, +) + + +def test_github_dot_com_deployment_preserves_default_contract_scope() -> None: + assert github_deployment_context("https://api.github.com") == ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + DEFAULT_GITHUB_WEB_ORIGIN, + ) + assert github_org_saml_service_provider_id("acme") == "github:saml:sp:org:acme" + assert ( + github_enterprise_saml_service_provider_id("acme") + == "github:saml:sp:enterprise:acme" + ) + + +def test_ghes_deployment_scopes_contract_ids_and_routes() -> None: + deployment_id, web_origin = github_deployment_context( + "https://api.github.example.com" + ) + + assert deployment_id == "api.github.example.com" + assert web_origin == "https://api.github.example.com" + assert ( + github_org_saml_service_provider_id("acme", deployment_id) + == "github:api.github.example.com:saml:sp:org:acme" + ) + assert github_org_acs_url("acme", web_origin) == ( + "https://api.github.example.com/orgs/acme/saml/consume" + ) + assert github_enterprise_acs_url("acme", web_origin) == ( + "https://api.github.example.com/enterprises/acme/saml/consume" + ) + + +@pytest.mark.parametrize("host", ["github.example.com", "ftp://github.example.com"]) +def test_github_deployment_context_rejects_invalid_hosts(host: str) -> None: + with pytest.raises(ValueError): + github_deployment_context(host) + + +def test_saml_match_helpers_preserve_source_exact_values() -> None: + attributes = [ + {"name": ENTRA_OBJECT_ID_CLAIM, "value": "object-1"}, + SimpleNamespace(name=ENTRA_OBJECT_ID_CLAIM, value="object-1"), + {"name": ENTRA_OBJECT_ID_CLAIM, "value": " object-2 "}, + {"name": "other", "value": "ignored"}, + ] + + assert saml_account_match_values(" Alice@example.com ", "", None, "Alice@example.com") == [ + "Alice@example.com" + ] + assert saml_attribute_match_values(attributes, ENTRA_OBJECT_ID_CLAIM) == [ + "object-1", + "object-2", + ] diff --git a/tests/test_saml_models.py b/tests/test_saml_models.py new file mode 100644 index 0000000..0f0ed67 --- /dev/null +++ b/tests/test_saml_models.py @@ -0,0 +1,278 @@ +from types import SimpleNamespace + +from openhound_github.kinds import edges as ek +from openhound_github.graphql import ENTERPRISE_SAML_QUERY, SAML_IDENTITIES_QUERY +from openhound_github.models.external_identity import ExternalIdentity +from openhound_github.models.saml_assertion_consumer_service import ( + SamlAssertionConsumerService, +) +from openhound_github.models.saml_helpers import ( + DEFAULT_GITHUB_DEPLOYMENT_ID, + DEFAULT_GITHUB_WEB_ORIGIN, + SAML_CONTRACT_VERSION, +) +from openhound_github.models.saml_issuer import SamlIssuer +from openhound_github.models.saml_provider import SamlProvider +from openhound_github.models.saml_service_provider import SamlServiceProvider + + +def _identity_with_lookup(**overrides) -> ExternalIdentity: + data = { + "guid": "guid-1", + "id": "external-identity-1", + "environment_slug": "acme", + "samlIdentity": { + "username": "Alice@example.com", + "nameId": "subject-1", + "attributes": [ + { + "name": "http://schemas.microsoft.com/identity/claims/objectidentifier", + "value": "object-1", + } + ], + }, + "scimIdentity": {"username": "scim-only@example.com"}, + "user": {"id": "USER_1", "login": "alice"}, + } + data.update(overrides) + identity = ExternalIdentity.model_validate(data) + identity._lookup = SimpleNamespace( + idp_for_environment=lambda _slug: [ + ( + "IDP_1", + "https://issuer.example.com", + "https://issuer.example.com/sso", + "ORG_1", + "Acme", + "org", + ) + ] + ) + return identity + + +def _saml_account_edge(identity: ExternalIdentity): + return next(edge for edge in identity.edges if edge.kind == ek.SAML_HAS_ACCOUNT) + + +def test_normalized_saml_nodes_expose_contract_metadata() -> None: + service_provider = SamlServiceProvider( + id="IDP_1", + issuer="https://issuer.example.com", + environment_node_id="ORG_1", + environment_name="Acme", + environment_slug="acme", + environment_type="org", + ) + issuer = SamlIssuer( + issuer="https://issuer.example.com", + environment_node_id="ORG_1", + environment_name="Acme", + environment_slug="acme", + environment_type="org", + ) + acs = SamlAssertionConsumerService( + environment_node_id="ORG_1", + environment_name="Acme", + environment_slug="acme", + environment_type="org", + ) + + sp_properties = service_provider.as_node.properties + issuer_properties = issuer.as_node.properties + acs_properties = acs.as_node.properties + + assert service_provider.as_node.id == "github:saml:sp:org:acme" + assert issuer.as_node.id == "github:saml:trusted-issuer:org:acme" + assert acs.as_node.id == "github:saml:acs:org:acme" + assert sp_properties.github_deployment_id == DEFAULT_GITHUB_DEPLOYMENT_ID + assert sp_properties.github_web_origin == DEFAULT_GITHUB_WEB_ORIGIN + assert sp_properties.schema_contract_version == SAML_CONTRACT_VERSION + + assert issuer_properties.github_deployment_id == DEFAULT_GITHUB_DEPLOYMENT_ID + assert issuer_properties.github_web_origin == DEFAULT_GITHUB_WEB_ORIGIN + assert issuer_properties.native_source_field == "GH_SamlIdentityProvider.issuer" + assert issuer_properties.schema_contract_version == SAML_CONTRACT_VERSION + + assert acs_properties.github_deployment_id == DEFAULT_GITHUB_DEPLOYMENT_ID + assert acs_properties.github_web_origin == DEFAULT_GITHUB_WEB_ORIGIN + assert acs_properties.route_source == "github_organization_scope_convention" + assert acs_properties.schema_contract_version == SAML_CONTRACT_VERSION + + implements_edges = list(service_provider.edges) + assert len(implements_edges) == 1 + assert implements_edges[0].start.value == "IDP_1" + assert implements_edges[0].end.value == "github:saml:sp:org:acme" + assert ( + implements_edges[0].properties.schema_contract_version + == SAML_CONTRACT_VERSION + ) + assert ( + next(iter(issuer.edges)).properties.schema_contract_version + == SAML_CONTRACT_VERSION + ) + assert ( + next(iter(acs.edges)).properties.schema_contract_version + == SAML_CONTRACT_VERSION + ) + + +def test_normalized_saml_nodes_scope_ids_and_routes_by_deployment() -> None: + service_provider = SamlServiceProvider( + id="IDP_1", + issuer="https://issuer.example.com", + environment_node_id="ORG_1", + environment_name="Acme", + environment_slug="acme", + environment_type="org", + github_deployment_id="github.example.com", + github_web_origin="https://github.example.com", + ) + issuer = SamlIssuer( + issuer="https://issuer.example.com", + environment_node_id="ORG_1", + environment_name="Acme", + environment_slug="acme", + environment_type="org", + github_deployment_id="github.example.com", + github_web_origin="https://github.example.com", + ) + acs = SamlAssertionConsumerService( + environment_node_id="ORG_1", + environment_name="Acme", + environment_slug="acme", + environment_type="org", + github_deployment_id="github.example.com", + github_web_origin="https://github.example.com", + ) + + assert service_provider.as_node.id == "github:github.example.com:saml:sp:org:acme" + assert issuer.as_node.id == "github:github.example.com:saml:trusted-issuer:org:acme" + assert acs.as_node.id == "github:github.example.com:saml:acs:org:acme" + assert acs.as_node.properties.acs_url == ( + "https://github.example.com/orgs/acme/saml/consume" + ) + assert service_provider.as_node.properties.entity_id == ( + "https://github.example.com/orgs/acme" + ) + + +def test_saml_identity_collection_requests_and_preserves_attributes() -> None: + assert "attributes {" in ENTERPRISE_SAML_QUERY + assert "attributes {" in SAML_IDENTITIES_QUERY + + identity = ExternalIdentity( + guid="guid-1", + id="external-identity-1", + environment_slug="acme", + samlIdentity={ + "username": "alice@example.com", + "attributes": [ + { + "name": "http://schemas.microsoft.com/identity/claims/objectidentifier", + "value": "object-1", + "metadata": "source-exact", + } + ], + }, + ) + + assert identity.saml_identity is not None + assert identity.saml_identity.attributes == [ + { + "name": "http://schemas.microsoft.com/identity/claims/objectidentifier", + "value": "object-1", + "metadata": "source-exact", + } + ] + + +def test_saml_provider_replays_snake_case_fields_and_deployment_metadata() -> None: + provider = SamlProvider.model_validate( + { + "id": "IDP_1", + "issuer": "https://issuer.example.com", + "sso_url": "https://issuer.example.com/sso", + "signature_method": "rsa-sha256", + "idp_certificate": "certificate-data", + "environment_node_id": "ORG_1", + "environment_name": "Acme", + "environment_slug": "acme", + "environment_type": "org", + "github_deployment_id": "github.example.com", + "github_web_origin": "https://github.example.com", + } + ) + + assert provider.sso_url == "https://issuer.example.com/sso" + assert provider.signature_method == "rsa-sha256" + assert provider.idp_certificate == "certificate-data" + assert provider.as_node.properties.github_deployment_id == "github.example.com" + assert provider.as_node.properties.github_web_origin == "https://github.example.com" + + +def test_saml_account_edge_emits_contract_evidence_without_changing_legacy_edges() -> None: + identity = _identity_with_lookup() + + edges = list(identity.edges) + account_edge = next(edge for edge in edges if edge.kind == ek.SAML_HAS_ACCOUNT) + + assert account_edge.properties.schema_contract_version == SAML_CONTRACT_VERSION + assert account_edge.properties.match_values == [ + "Alice@example.com", + "subject-1", + "object-1", + ] + assert account_edge.properties.scoped_exact_match_values == [ + "Alice@example.com", + "subject-1", + ] + assert account_edge.properties.entra_object_id_match_values == ["object-1"] + assert account_edge.properties.direct_binding is True + assert account_edge.properties.direct_binding_source == ( + "GH_ExternalIdentity.saml_identity" + ) + assert account_edge.properties.external_identity_id == "external-identity-1" + assert account_edge.start.value == "github:saml:sp:org:acme" + + assert ek.HAS_EXTERNAL_IDENTITY in {edge.kind for edge in edges} + assert ek.MAPS_TO_USER in {edge.kind for edge in edges} + + +def test_org_scim_only_identity_does_not_emit_saml_account_edge() -> None: + identity = _identity_with_lookup( + samlIdentity=None, + scimIdentity={"username": "scim-only@example.com"}, + ) + + assert ek.SAML_HAS_ACCOUNT not in {edge.kind for edge in identity.edges} + + +def test_enterprise_scim_only_identity_emits_managed_user_binding() -> None: + identity = _identity_with_lookup( + samlIdentity=None, + scimIdentity={"username": "managed@example.com"}, + ) + identity._lookup = SimpleNamespace( + idp_for_environment=lambda _slug: [ + ( + "IDP_1", + "https://issuer.example.com", + "https://issuer.example.com/sso", + "ENT_1", + "Acme", + "enterprise", + ) + ] + ) + + account_edge = _saml_account_edge(identity) + + assert account_edge.properties.match_values == ["managed@example.com"] + assert account_edge.properties.scoped_exact_match_values == [ + "managed@example.com" + ] + assert account_edge.properties.entra_object_id_match_values == [] + assert account_edge.properties.direct_binding_source == ( + "GH_ExternalIdentity.scim_identity (Enterprise Managed Users)" + ) diff --git a/tests/test_scim_models.py b/tests/test_scim_models.py new file mode 100644 index 0000000..72663db --- /dev/null +++ b/tests/test_scim_models.py @@ -0,0 +1,162 @@ +from unittest.mock import MagicMock + +import duckdb +from openhound.core.models.entries_dataclass import ConditionalEdgePath + +from openhound_github.kinds import edges as ek +from openhound_github.kinds import nodes as nk +from openhound_github.lookup import GithubLookup +from openhound_github.models import EnterpriseTeam, ScimGroup, ScimOrganization, ScimUser +from openhound_github.resources.enterprise import iter_enterprise_scim_resources + + +def _scim_user(*, legacy: bool = False) -> ScimUser: + return ScimUser( + id="scim-user-1", + externalId="00u-okta-1", + userName="alice@example.test", + displayName="Alice Example", + name={"givenName": "Alice", "familyName": "Example"}, + emails=[{"value": "alice@example.test", "primary": True}], + active=True, + enterprise_node_id="ENT_NODE_1", + enterprise_slug="example-enterprise", + emit_legacy_correlation=legacy, + ) + + +def test_enterprise_scim_uses_enterprise_endpoint_and_scim_pagination() -> None: + client = MagicMock() + client.paginate.return_value = [[{"id": "u1"}], [{"id": "u2"}]] + + rows = list( + iter_enterprise_scim_resources(client, "example-enterprise", "Users") + ) + + assert rows == [{"id": "u1"}, {"id": "u2"}] + args, kwargs = client.paginate.call_args + assert args[0] == "/scim/v2/enterprises/example-enterprise/Users" + assert kwargs["params"] == {"startIndex": 1, "count": 100} + assert kwargs["data_selector"] == "Resources" + assert kwargs["paginator"].param_name == "startIndex" + assert kwargs["paginator"].initial_value == 1 + assert kwargs["paginator"].limit_param == "count" + + +def test_scim_user_emits_normalized_edges_without_legacy_correlation_by_default() -> None: + user = _scim_user() + + node = user.as_node + edges = list(user.edges) + + assert node.kinds == [nk.SCIM_USER] + assert node.properties.environmentid == "ENT_NODE_1" + assert node.properties.external_id == "00u-okta-1" + assert [edge.kind for edge in edges] == [ + ek.SCIM_CONTAINS, + ek.SCIM_PROVISIONED, + ] + assert isinstance(edges[1].end, ConditionalEdgePath) + assert edges[1].end.kind == nk.EXTERNAL_IDENTITY + assert edges[1].end.property_matchers[0].key == "guid" + assert edges[1].end.property_matchers[0].value == "scim-user-1" + + +def test_scim_user_legacy_idp_correlation_is_explicitly_gated() -> None: + edges = list(_scim_user(legacy=True).edges) + + legacy_edge = edges[-1] + assert legacy_edge.kind == ek.SCIM_PROVISIONED + assert legacy_edge.start.value == "00u-okta-1" + assert legacy_edge.end.value == "scim-user-1" + assert legacy_edge.properties.traversable is True + + +def test_scim_group_emits_membership_and_tenant_scoped_legacy_correlation() -> None: + group = ScimGroup( + id="scim-group-1", + externalId="Engineering", + displayName="Engineering", + members=[{"value": "scim-user-1"}, {"value": "scim-user-2"}], + enterprise_node_id="ENT_NODE_1", + enterprise_slug="example-enterprise", + emit_legacy_correlation=True, + ) + lookup = MagicMock() + lookup.enterprise_idp_for_scope.return_value = ( + "http://www.okta.com/exk-example", + "https://preview2.example.okta.com/app/github/sso/saml", + ) + group._lookup = lookup + + edges = list(group.edges) + + assert [edge.kind for edge in edges] == [ + ek.SCIM_CONTAINS, + ek.SCIM_MEMBER_OF, + ek.SCIM_MEMBER_OF, + ek.SCIM_PROVISIONED, + ] + legacy_edge = edges[-1] + assert isinstance(legacy_edge.start, ConditionalEdgePath) + assert { + matcher.key: matcher.value for matcher in legacy_edge.start.property_matchers + } == { + "tenant_domain": "preview2.example.okta.com", + "name": "ENGINEERING", + } + + +def test_enterprise_team_emits_scim_provisioning_edge_with_group_id() -> None: + team = EnterpriseTeam( + id=7, + name="Engineering", + slug="engineering", + group_id="scim-group-1", + enterprise_node_id="ENT_NODE_1", + enterprise_slug="example-enterprise", + ) + lookup = MagicMock() + lookup.enterprise_id.return_value = "ENT_NODE_1" + team._lookup = lookup + + edges = list(team.edges) + + assert [edge.kind for edge in edges] == [ek.CONTAINS, ek.SCIM_PROVISIONED] + assert edges[1].start.value == "scim-group-1" + + +def test_scim_organization_stays_within_github_environment_root() -> None: + organization = ScimOrganization( + enterprise_node_id="ENT_NODE_1", + enterprise_slug="example-enterprise", + ) + + node = organization.as_node + + assert node.id == "SCIM_Organization_ENT_NODE_1" + assert node.kinds == [nk.SCIM_ORGANIZATION] + assert node.properties.environmentid == "ENT_NODE_1" + + +def test_enterprise_idp_lookup_uses_unified_saml_provider_table() -> None: + connection = duckdb.connect(":memory:") + connection.execute("CREATE SCHEMA github") + connection.execute( + """CREATE TABLE github.saml_provider ( + environment_node_id VARCHAR, + environment_type VARCHAR, + issuer VARCHAR, + sso_url VARCHAR + )""" + ) + connection.execute( + """INSERT INTO github.saml_provider VALUES + ('ENT_NODE_1', 'enterprise', 'http://www.okta.com/exk-example', + 'https://preview2.example.okta.com/app/github/sso/saml')""" + ) + + assert GithubLookup(connection).enterprise_idp_for_scope("ENT_NODE_1") == ( + "http://www.okta.com/exk-example", + "https://preview2.example.okta.com/app/github/sso/saml", + ) diff --git a/tests/test_workflow_model.py b/tests/test_workflow_model.py new file mode 100644 index 0000000..31b4ae6 --- /dev/null +++ b/tests/test_workflow_model.py @@ -0,0 +1,63 @@ +import base64 +from datetime import datetime +from unittest.mock import MagicMock + +from openhound_github.kinds import edges as ek +from openhound_github.models.workflow import Workflow + + +def _make_pwn_request_workflow() -> Workflow: + contents = base64.b64encode( + b"""on: + pull_request_target: +jobs: + build: + steps: + - uses: actions/checkout@v4 + with: + ref: ${{ github.event.pull_request.head.sha }} +""" + ).decode() + workflow = Workflow( + id=1, + node_id="W_1", + name="pr.yml", + path=".github/workflows/pr.yml", + state="active", + created_at=datetime(2026, 1, 1), + updated_at=datetime(2026, 1, 1), + url="https://api.github.test/repos/org/repo/actions/workflows/1", + contents=contents, + org_login="org", + repository_name="repo", + repository_node_id="R_1", + ) + lookup = MagicMock() + lookup.repository_allow_forking.return_value = ("public", True) + lookup.repo_role_node_ids_with_read_repo_contents.return_value = [ + ("ROLE_1",), + ("ROLE_2",), + ] + lookup.branches_for_repository.return_value = [ + ("B_main", "main", False), + ("B_release", "release/v1", True), + ] + workflow._lookup = lookup + return workflow + + +def test_pwn_request_edges_support_branch_lookup_protection_flag() -> None: + workflow = _make_pwn_request_workflow() + + edges = [ + edge for edge in workflow._can_pwn_request_edges if edge.kind == ek.CAN_PWN_REQUEST + ] + + assert {(edge.start.value, edge.end.value) for edge in edges} == { + ("ROLE_1", "R_1"), + ("ROLE_1", "B_main"), + ("ROLE_1", "B_release"), + ("ROLE_2", "R_1"), + ("ROLE_2", "B_main"), + ("ROLE_2", "B_release"), + }