Deze documentatie wordt actief uitgebreid — kom regelmatig terug.

CTN Endpoint Lifecycle

Overview

This document describes the complete lifecycle of a data endpoint within the Connected Trade Network (CTN) — from registration by a provider to data retrieval by a consumer. Keycloak is used as the identity and token service for all M2M (machine-to-machine) authentication.

Terminology

Term	Definition
Provider	Organization that makes data available via an endpoint (data owner)
Consumer	Organization that retrieves data from a provider
Endpoint	An API/data source operated by a provider from which data can be retrieved
System	A technical system (application/service) that operates on behalf of an organization. Can be single-tenant (own system) or multi-tenant (shared, e.g., a SaaS platform)
ASR	Association Register — the system that manages association participation, performs compliance checks, and issues VAD tokens. In the endpoint lifecycle, the ASR also manages registrations, relations, and access grants
M2M Client	A Keycloak OAuth 2.0 client using the Client Credentials grant, used for system-to-system communication
Keycloak	Open-source identity provider used by CTN for authentication, token issuance, and M2M client management

Lifecycle Overview

flowchart LR
    subgraph provider["PROVIDER (data owner)"]
        direction LR
        R["1 Registration<br/>Enter endpoint URL"] --> V["2 Validation<br/>Prove ownership"]
        V --> P["3 Publication<br/>Visible in service directory<br/>+ set access model"]
    end

    subgraph consumer["CONSUMER (data user)"]
        direction LR
        A["4 Request Access<br/>Register system in ASR,<br/>create M2M client,<br/>request endpoint access"] --> AU{"5 Authorization"}
        AU -->|"open: auto-approved"| D["6 Data Exchange<br/>Fetch token from<br/>Keycloak, retrieve data"]
        AU -->|"request: provider approves"| D
        AU -->|"restricted: invite only"| D
    end

    P --> A

    style provider fill:#e8f4fd,stroke:#1a73e8,stroke-width:2px
    style consumer fill:#e8f8e8,stroke:#34a853,stroke-width:2px
    style R fill:#fff,stroke:#1a73e8
    style V fill:#fff,stroke:#1a73e8
    style P fill:#fff,stroke:#1a73e8
    style A fill:#fff,stroke:#34a853
    style AU fill:#fff3cd,stroke:#e6a800
    style D fill:#fff,stroke:#34a853

1. Registration

The provider registers an endpoint in the CTN Member Portal.

Fields captured:

Field	Description
`endpoint_url`	Base URL of the endpoint (e.g., `https://api.company.nl/ctn`)
`name`	Human-readable name for the endpoint
`description`	Description of the available data
`data_type`	Type of data the endpoint serves (e.g., `container_status`, `eta`, `transport_order`)
`provider_org_id`	The organization that owns the endpoint

Status after this step: REGISTERED

2. Validation

After registration, the provider must prove they actually control the endpoint. This is done through a Callback Challenge-Response mechanism (industry standard, used by Stripe, GitHub, Slack).

Wizard Flow

Step	Action
1. Details	Provider enters endpoint URL
2. Verify	CTN sends challenge to endpoint → auto-verified
3. Test	Health check (`GET /health`) confirms connectivity
4. Activate	Endpoint status set to `VERIFIED`

Sequence Diagram

sequenceDiagram
    autonumber
    participant M as Provider<br/>(Portal UI)
    participant CTN as CTN Backend
    participant E as Provider's<br/>Endpoint

    rect rgb(240, 248, 255)
        Note over M,E: Step 1 — Register Endpoint
        M->>CTN: Register endpoint URL<br/>POST /api/endpoints
        CTN->>CTN: Validate URL format<br/>Generate challenge token<br/>(UUID + HMAC signature)
        CTN-->>M: 202 Accepted<br/>Status: "Verifying..."
    end

    rect rgb(240, 255, 240)
        Note over M,E: Step 2 — Challenge-Response Verification
        CTN->>E: POST {endpoint_url}/.well-known/ctn/verify<br/>Headers: X-CTN-Signature<br/>Body: { "challenge": "abc123...", "timestamp": "..." }
        
        alt ✅ Endpoint responds correctly
            E-->>CTN: 200 OK<br/>{ "challenge": "abc123..." }
            CTN->>CTN: Validate challenge match<br/>+ response time < 10s
            CTN-->>M: ✅ Verification Passed
        else ❌ Wrong challenge / timeout / error
            E-->>CTN: 4xx / 5xx / timeout
            CTN-->>M: ❌ Verification Failed<br/>Show error details
            Note over M: Provider can retry<br/>after fixing endpoint
        end
    end

    rect rgb(255, 248, 240)
        Note over M,E: Step 3 — Connection Test
        CTN->>E: GET {endpoint_url}/health<br/>Headers: Authorization (Bearer)
        E-->>CTN: 200 OK<br/>{ "status": "healthy", "version": "1.0" }
        CTN-->>M: ✅ Connection Test Passed
    end

    rect rgb(248, 240, 255)
        Note over M,E: Step 4 — Activate
        M->>CTN: Activate endpoint
        CTN->>CTN: Status: VERIFIED
        CTN-->>M: ✅ Endpoint verified
    end

Primary: Callback Challenge-Response

CTN → Provider Endpoint

POST {endpoint_url}/.well-known/ctn/verify
Content-Type: application/json
X-CTN-Signature: sha256=<hmac_of_body>
X-CTN-Request-Id: <uuid>

{
  "challenge": "ctn_v1_a3b8f2e1-4c6d-4e8f-9a1b-2c3d4e5f6789",
  "timestamp": "2026-02-04T09:30:00Z",
  "endpoint_id": "ep_abc123"
}

Expected response (within 10 seconds)

HTTP/1.1 200 OK
Content-Type: application/json

{
  "challenge": "ctn_v1_a3b8f2e1-4c6d-4e8f-9a1b-2c3d4e5f6789"
}

Validation rules:

Response must be HTTP 200
challenge value must match exactly
Response must arrive within 10 seconds
X-CTN-Signature header allows the provider to verify the request genuinely came from CTN

Challenge token format:

ctn_v1_{uuid}

Prefix ctn_v1_ for versioning
UUID v4 for uniqueness
Single-use, expires after 15 minutes
HMAC-SHA256 signature of the body using a shared signing secret

Provider Implementation Requirement

The provider implements a single endpoint:

POST /.well-known/ctn/verify

Minimal implementation (illustrative; the actual Quarkus version lives in the ASR codebase):

app.post('/.well-known/ctn/verify', (req, res) => {
  const { challenge } = req.body;
  // Optionally verify X-CTN-Signature header
  res.json({ challenge });
});

Fallback: Well-Known URL Verification

If the callback challenge fails (e.g., firewall blocks inbound POST, or the verify handler hasn’t been deployed yet), CTN falls back to file-based verification.

CTN generates a verification token
Provider places a JSON file at a well-known path on their endpoint domain
CTN fetches the file to confirm the provider controls the server

Location:

GET {endpoint_url}/.well-known/ctn/verification.json

Expected content:

{
  "ctn_verification": "ctn_v1_a3b8f2e1-4c6d-4e8f-9a1b-2c3d4e5f6789",
  "endpoint_id": "ep_abc123",
  "created_at": "2026-02-04T09:30:00Z"
}

File must be accessible via HTTPS
CTN polls up to 3 times at 30-second intervals
Token expires after 24 hours

Status after this step: VERIFIED

3. Publication

After successful validation, the provider publishes the endpoint, making it visible in the CTN service directory for other participants.

What the Provider Configures

Setting	Description
Access model	`open`, `request`, or `restricted` (see below)
Scopes	Which data operations are available (e.g., `read:container_status`, `read:eta`)
Rate limits	Maximum requests per consumer per time period
Description	Documentation about the endpoint for potential consumers

Access Models

The access model determines how consumers gain access to the endpoint. This choice affects the flow through steps 4 and 5.

Model	Behavior	Steps 4d + 5
`open`	Any registered CTN member can access the endpoint immediately. No approval required.	Consumer requests access → auto-approved instantly. Keycloak scopes and policies are created automatically.
`request`	Consumers must request access. The provider reviews and approves or denies each request.	Consumer requests access → status `PENDING` → provider approves/denies → Keycloak updated on approval.
`restricted`	Only organizations explicitly invited by the provider can access the endpoint. The endpoint is visible in the directory but the “Request Access” button is hidden for non-invited organizations.	Provider sends invite → consumer accepts → auto-approved.

Publication Action

The provider clicks “Publish” in the Member Portal. CTN registers the endpoint in the service directory.

POST /api/endpoints/{endpoint_id}/publish
Authorization: Bearer {provider_token}

{
  "access_model": "open",
  "available_scopes": ["read:container_status", "read:eta"],
  "rate_limit": { "requests_per_minute": 60 },
  "description": "Real-time container status updates for terminal X"
}

Status after this step: PUBLISHED

The endpoint is now discoverable in the service directory. For open endpoints, consumers can immediately request and receive access. For request and restricted endpoints, additional steps are required.

4. Request Access

A consumer that wants to retrieve data from a published endpoint goes through the following steps.

4a. Register System in ASR

Before a consumer can request access, the system they use to retrieve data must be registered in the ASR.

Field	Description
`system_name`	Name of the system
`system_type`	`single_tenant` (own system) or `multi_tenant` (SaaS/shared platform)
`owner_org_id`	Organization that manages the system

Single-tenant systems can only be used by the owning organization. Multi-tenant systems (e.g., a SaaS platform from a software vendor) can be linked by multiple organizations.

Design decision: Software vendors offering multi-tenant systems are registered as organizations in the ASR with the role system_provider. They register the system; participants then create a relation with that system.

4b. Create Relation: Organization ↔ System

A consumer organization registers in the ASR that a specific system may operate on its behalf.

POST /asr/relations
Authorization: Bearer {consumer_org_token}

{
  "organization_id": "org_consumer_456",
  "system_id": "sys_tms_789",
  "relation_type": "authorized_consumer"
}

This means: “System X may retrieve data on behalf of Organization Y.”

4c. Create M2M Client in Keycloak

When a system is registered in the ASR and linked to an organization, a corresponding M2M client is created in Keycloak. This happens automatically via the CTN backend or can be triggered via the Member Portal (”+ Add M2M Client”).

Keycloak Configuration: M2M Client

Realm: ctn

Client settings:

Setting	Value
Client ID	`m2m-{org_id}-{system_id}` (e.g., `m2m-org_consumer_456-sys_tms_789`)
Client Protocol	`openid-connect`
Access Type	`confidential`
Service Accounts Enabled	`true` (required for Client Credentials grant)
Standard Flow Enabled	`false`
Direct Access Grants Enabled	`false`
Authorization Enabled	`true`

Client Credentials:

After creation, Keycloak generates a client_id and client_secret. These are presented once to the consumer in the Member Portal and must be stored securely by the consumer.

Client Scopes:

Each M2M client is assigned scopes that match the granted endpoint access. Scopes follow the pattern:

endpoint:{endpoint_id}:{operation}

Examples: endpoint:ep_abc123:read, endpoint:ep_abc123:write

Keycloak Configuration: Custom Token Mapper

To include organization and system context in the JWT, configure a custom protocol mapper on the M2M client:

Mapper Setting	Value
Name	`ctn-org-claims`
Mapper Type	`Hardcoded claim` or `Script Mapper`
Token Claim Name	`ctn_org_id`
Claim Value	`{organization_id}`
Add to access token	`true`

Additional claims to include:

Claim	Value	Purpose
`ctn_org_id`	Organization ID	Identifies which organization the token represents
`ctn_system_id`	System ID	Identifies which system is making the request
`ctn_system_type`	`single_tenant` / `multi_tenant`	Allows the provider to distinguish system types

4d. Request Access to Endpoint

The consumer requests access to a published endpoint via the Member Portal.

POST /api/endpoints/{endpoint_id}/access-requests
Authorization: Bearer {consumer_token}

{
  "consumer_org_id": "org_consumer_456",
  "requested_scopes": ["read:container_status"],
  "purpose": "Real-time tracking for our supply chain"
}

What happens next depends on the access model:

Access model	Result
`open`	Access request is auto-approved. Keycloak scopes and authorization policies are created immediately. Consumer receives a `201 Created` with status `APPROVED`.
`request`	Access request status is set to `PENDING`. Provider is notified. See step 5.
`restricted`	If the consumer was invited by the provider, access is auto-approved. Otherwise, the request is rejected with `403 Forbidden`.

5. Authorization

This step differs based on the access model configured by the provider.

Access Model: `open`

No manual authorization step. When the consumer requests access (step 4d), the CTN backend immediately:

Creates the AccessGrant with status APPROVED
Assigns the endpoint scopes to the consumer’s Keycloak M2M client
Creates the Keycloak authorization policy and permission

The consumer can proceed directly to step 6 (Data Exchange).

Access Model: `request`

The provider receives a notification that a consumer has requested access and reviews the request in the Member Portal.

Provider Approves or Denies:

PUT /api/access-requests/{request_id}
Authorization: Bearer {provider_token}

{
  "decision": "approved",
  "granted_scopes": ["read:container_status"],
  "expires_at": "2027-02-04T00:00:00Z"
}

Access Model: `restricted`

The provider proactively invites specific organizations via the Member Portal:

POST /api/endpoints/{endpoint_id}/invitations
Authorization: Bearer {provider_token}

{
  "invited_org_id": "org_partner_789",
  "granted_scopes": ["read:container_status"],
  "expires_at": "2027-02-04T00:00:00Z"
}

When the invited consumer accepts (or requests access), authorization is auto-approved.

What Happens After Approval

The ASR registers the authorization: consumer organization X may use scopes Z on endpoint E
The CTN backend updates the Keycloak M2M client for the consumer, adding the granted scopes
The Keycloak authorization policy is updated to allow token issuance for this endpoint

Keycloak Configuration: Authorization Policy

For each approved access grant, an authorization policy is configured in Keycloak:

Resource:

Setting	Value
Name	`endpoint-{endpoint_id}`
Type	`ctn:endpoint`
URI	`{endpoint_url}/*`
Scopes	`read`, `write` (as applicable)

Policy:

Setting	Value
Name	`policy-{consumer_org_id}-{endpoint_id}`
Type	`Client Policy`
Clients	`m2m-{org_id}-{system_id}`
Logic	`Positive`

Permission:

Setting	Value
Name	`perm-{consumer_org_id}-{endpoint_id}`
Type	`Scope-based`
Resource	`endpoint-{endpoint_id}`
Scopes	Granted scopes (e.g., `read`)
Policies	`policy-{consumer_org_id}-{endpoint_id}`

Status after this step: Access request APPROVED

6. Data Exchange

With the full trust path established, the consumer system can now retrieve data from the provider endpoint.

Token Flow via Keycloak

sequenceDiagram
    autonumber
    participant CS as Consumer<br/>System
    participant KC as Keycloak<br/>(ASR Token Service)
    participant ASR as ASR<br/>(CTN Backend)
    participant PE as Provider<br/>Endpoint

    rect rgb(240, 248, 255)
        Note over CS,PE: Token Request (OAuth 2.0 Client Credentials)
        CS->>KC: POST /realms/ctn/protocol/openid-connect/token<br/>grant_type=client_credentials<br/>client_id={m2m_client_id}<br/>client_secret={m2m_client_secret}<br/>scope=endpoint:{endpoint_id}:read
        KC->>ASR: Validate via custom SPI/policy:<br/>1. System registered?<br/>2. Organization registered?<br/>3. System↔Organization relation?<br/>4. Access to endpoint approved?
        
        alt ✅ All checks pass
            ASR-->>KC: Approved + claims
            KC-->>CS: 200 OK<br/>{ "access_token": "eyJ...",<br/>"token_type": "Bearer",<br/>"expires_in": 3600 }
        else ❌ Check fails
            ASR-->>KC: Denied
            KC-->>CS: 403 Forbidden<br/>{ "error": "access_denied" }
        end
    end

    rect rgb(240, 255, 240)
        Note over CS,PE: Data Retrieval
        CS->>PE: GET {endpoint_url}/data<br/>Authorization: Bearer {access_token}
        PE->>PE: Validate JWT:<br/>- Signature (Keycloak public key)<br/>- Claims (org_id, scopes)<br/>- Expiry
        PE-->>CS: 200 OK<br/>{ data }
    end

Token Request

The consumer system requests a token using the standard OAuth 2.0 Client Credentials flow:

POST /realms/ctn/protocol/openid-connect/token
Content-Type: application/x-www-form-urlencoded

grant_type=client_credentials
&client_id=m2m-org_consumer_456-sys_tms_789
&client_secret={client_secret}
&scope=endpoint:ep_abc123:read

What Keycloak/ASR Validates at Token Issuance

System registered? — Is the consumer system known in the ASR?
Organization registered? — Is the consumer organization known in the ASR?
System ↔ Organization relation? — Is this system authorized to operate on behalf of this organization?
Access to endpoint? — Has the provider granted this consumer organization access to this endpoint?

Only when all four checks pass does Keycloak issue an access token.

JWT Token Claims

The issued JWT contains the following CTN-specific claims:

{
  "iss": "https://keycloak.ctn.network/realms/ctn",
  "sub": "m2m-org_consumer_456-sys_tms_789",
  "aud": "ctn-api",
  "exp": 1738666200,
  "iat": 1738662600,
  "scope": "endpoint:ep_abc123:read",
  "ctn_org_id": "org_consumer_456",
  "ctn_system_id": "sys_tms_789",
  "ctn_system_type": "single_tenant"
}

What the Provider Validates on a Data Request

The provider only needs to validate the token — all trust logic resides in the ASR/Keycloak. The provider checks:

Token signature — Is the token valid and signed by Keycloak? (Verified using Keycloak’s public key, available at /realms/ctn/protocol/openid-connect/certs)
Token claims — Does the ctn_org_id match an authorized consumer?
Scopes — Is the consumer only requesting data they’re authorized for?
Token expiry — Is the token still valid?

This is the core simplification: complexity is removed from both provider and consumer and centralized in Keycloak/ASR.

ASR Entities

erDiagram
    Organization ||--o{ System : "operates via"
    Organization ||--o{ Endpoint : "owns (as provider)"
    Organization ||--o{ AccessGrant : "has (as consumer)"
    System ||--o{ M2MClient : "has"
    Endpoint ||--o{ AccessGrant : "grants access to"

    Organization {
        string org_id PK
        string name
        string kvk_number
        enum role "provider / consumer / system_provider"
    }

    System {
        string system_id PK
        string name
        enum type "single_tenant / multi_tenant"
        string owner_org_id FK
    }

    Endpoint {
        string endpoint_id PK
        string url
        string provider_org_id FK
        enum status "registered / verified / published / active / suspended / revoked"
        string[] scopes
        enum access_model "open / request / restricted"
    }

    AccessGrant {
        string grant_id PK
        string consumer_org_id FK
        string endpoint_id FK
        string[] granted_scopes
        datetime expires_at
        enum status "pending / approved / denied / revoked"
    }

    M2MClient {
        string client_id PK
        string system_id FK
        string org_id FK
        string keycloak_client_id
        string[] scopes
    }

Endpoint Status Lifecycle

stateDiagram-v2
    [*] --> REGISTERED
    REGISTERED --> VERIFIED : Challenge-response passed
    VERIFIED --> PUBLISHED : Provider publishes
    PUBLISHED --> ACTIVE : First consumer granted access
    ACTIVE --> SUSPENDED : Temporarily unavailable
    SUSPENDED --> ACTIVE : Reactivated
    SUSPENDED --> REVOKED : Permanently removed
    ACTIVE --> REVOKED : Permanently removed
    REVOKED --> [*]

Status	Meaning
`REGISTERED`	URL submitted, not yet verified
`VERIFIED`	Ownership proven via challenge-response
`PUBLISHED`	Visible in service directory, consumers can request access
`ACTIVE`	At least one consumer has active access
`SUSPENDED`	Temporarily unavailable (by provider or by CTN)
`REVOKED`	Permanently removed from the service directory

Keycloak Configuration Summary

Realm Setup

Setting	Value
Realm name	`ctn`
Token lifespan (access token)	`3600` seconds (1 hour)
Refresh token	Disabled for M2M clients
JWKS endpoint	`/realms/ctn/protocol/openid-connect/certs`

Client Scopes (Realm-level)

Create the following client scopes in Keycloak at the realm level. These are assigned to M2M clients as access is granted.

Scope Name	Description
`endpoint:{endpoint_id}:read`	Read access to a specific endpoint
`endpoint:{endpoint_id}:write`	Write access to a specific endpoint (if applicable)

Scopes are created dynamically by the CTN backend when an endpoint is published, and assigned to M2M clients when access is granted.

Provider Token Validation

Providers validate incoming JWTs by fetching Keycloak’s public keys:

GET https://keycloak.ctn.network/realms/ctn/protocol/openid-connect/certs

Most HTTP frameworks and API gateways support automatic JWKS-based validation. The provider does not need to call the ASR or Keycloak at request time — token validation is done locally using the cached public key.

Security Considerations

Request signing: All CTN outbound requests include X-CTN-Signature (HMAC-SHA256)
Token expiry: Challenge tokens and access tokens are time-limited
Single-use tokens: Each verification attempt generates a fresh token
Rate limiting: Maximum 5 verification attempts per endpoint per hour
HTTPS only: Endpoints must use TLS — no plain HTTP allowed
IP allowlisting (optional): Providers can restrict the verify endpoint to CTN’s egress IPs
OAuth 2.0 Client Credentials: M2M communication uses standard OAuth 2.0 flow via Keycloak
Scope-based access: Consumers only get access to specific data operations
JWT local validation: Providers validate tokens locally using Keycloak’s public keys — no runtime dependency on ASR

In samenwerking met