LOE: 13 (L) | Rationale: Two critical architectural bugs (circular loop prevention + event ordering) require cross-cutting changes to the observer layer, the legacy NetSuite sync service, and the canonical mapper. High-severity envelope field gaps require coordinated changes across the job, mapper, and observer. Medium gaps include a documented architectural decision on outbox deferral. Combined, this is a 20–40h effort with non-trivial regression risk.

# Jira Task: Refactor

**Title:** [Epic 4 Gap Resolution] Fix Shadow Sync Publisher — Canonical Envelope, Circular Loop Prevention & Event Ordering
**Priority:** Critical
**Story Points:** 13
**Assignee:** TBD

## Context

During gap analysis of the Epic 4 implementation against the [NetSuite Sync design spec](https://github.com/SuiteDynamics/SuiteX/wiki/Design-%7C-NetSuite-Sync-%E2%80%94-Design,-Requirements,-Deliverables,-and-Runbook), 12 defects were found spanning two critical bugs, three high-severity envelope omissions, four medium correctness issues, and three low-severity cleanups. The two critical gaps are active data-correctness risks: (1) shadow events dispatched before the legacy NetSuite write completes — if the write fails, ghost events exist in `events.raw`; (2) outbound NetSuite payloads have no marker fields, causing the NetSuite UE script to treat SuiteX writes as organic changes and emit back a new event, creating an infinite circular update loop.

### Current System Behavior

- `ProjectObserver` / `ProjectTaskObserver` dispatch `FormatAndPublishSyncEvent` on Eloquent `created`/`updated`/`deleted` hooks (`$afterCommit = true`) — before the legacy NetSuite API call.
- Legacy NetSuite sync service sends payloads with no `custbody_suitex_write_id` or `custbody_suitex_write_source` marker fields.
- `CanonicalEventMapper::buildEnvelope()` is missing `orderingKey`, `actorId`, `baseVersion`, and `transactionGroupId`.
- The envelope field is named `eventType` where the [canonical envelope schema](https://github.com/SuiteDynamics/SuiteX/wiki/Design-%7C-NetSuite-Sync-%E2%80%94-Design,-Requirements,-Deliverables,-and-Runbook#c1-changeevent-schema) requires `operation`.
- `updated` events emit the full model snapshot instead of only dirty fields (`getDirty()`).
- `deleted` events emit a non-canonical tombstone: `{ "writeId": "...", "sourceSystem": "suitex", "action": "deleted", "id": 1234 }` — not a full canonical envelope.
- `startDate` in the ProjectPayload schema is declared `"format": "date-time"` while the mapper produces `YYYY-MM-DD` (`"format": "date"`).
- Field mappings are hardcoded inline without an extractable `getFieldMap()` method.
- Observer + Redis queue is silent-loss-prone; no transactional outbox pattern.

- **Relevant Files:**
  - `app/Domain/DataSync/Observers/ProjectObserver.php`
  - `app/Domain/DataSync/Observers/ProjectTaskObserver.php`
  - `app/Domain/DataSync/Services/CanonicalEventMapper.php`
  - `app/Domain/DataSync/Jobs/FormatAndPublishSyncEvent.php`
- **Current Pattern:** Observers dispatch the shadow event job directly on Eloquent lifecycle hooks, independent of whether the downstream NetSuite HTTP call succeeds.

## Description

Deliver a corrected Shadow Sync Publisher that: (1) dispatches shadow events only after a confirmed successful NetSuite write, (2) injects circular-loop-prevention markers into every outbound NetSuite payload, (3) emits a complete canonical envelope including `orderingKey`, `actorId`, `baseVersion`, and `transactionGroupId`, (4) uses the field name `operation` throughout, (5) emits field-level deltas on `update` events via `getChanges()` (post-save dirty tracking), (6) emits a full canonical envelope with `operation: "delete"` and `changes: {}` for deletes (no custom tombstone), and (7) documents the transactional outbox migration path.

### Areas to Review

- `app/Domain/DataSync/Observers/ProjectObserver.php` — remove `FormatAndPublishSyncEvent::dispatch()` calls; observer becomes a no-op for the shadow event path
- `app/Domain/DataSync/Observers/ProjectTaskObserver.php` — same as above
- `app/Domain/DataSync/Services/CanonicalEventMapper.php` — primary target for all envelope corrections
- `app/Domain/DataSync/Jobs/FormatAndPublishSyncEvent.php` — accept `$actorId` in constructor; rename `$eventType` → `$operation`
- Legacy NetSuite project sync service — add `$writeId` generation, marker injection, and post-success shadow event dispatch
- Legacy NetSuite project task sync service — same as above
- [Legacy API Coexistence & Shadow Events](https://github.com/SuiteDynamics/SuiteX/wiki/Design-%7C-NetSuite-Sync-%E2%80%94-Design,-Requirements,-Deliverables,-and-Runbook#legacy-api-coexistence--shadow-events-strangler-fig-pattern), [Circular Update Prevention](https://github.com/SuiteDynamics/SuiteX/wiki/Design-%7C-NetSuite-Sync-%E2%80%94-Design,-Requirements,-Deliverables,-and-Runbook#circular-update-prevention--event-source-attribution), [Appendix C1 Canonical Envelope Schema](https://github.com/SuiteDynamics/SuiteX/wiki/Design-%7C-NetSuite-Sync-%E2%80%94-Design,-Requirements,-Deliverables,-and-Runbook#c1-changeevent-schema)

## Deliverables

1. Remove `FormatAndPublishSyncEvent::dispatch()` from `ProjectObserver` and `ProjectTaskObserver`. Move dispatch to the legacy NetSuite sync service, called after a confirmed `2xx` response with `internalId`.
2. Generate `$writeId = (string) Str::uuid()` before the legacy NetSuite HTTP call; inject `custbody_suitex_write_id` and `custbody_suitex_write_source = 'suitex'` into the outbound payload; pass `$writeId` to the shadow event after success.
3. Add `orderingKey`, `actorId`, `baseVersion`, and `transactionGroupId` to `CanonicalEventMapper::buildEnvelope()`.
4. Capture `auth()->id()` at dispatch time (sync service call site) and pass it via the `FormatAndPublishSyncEvent` constructor. The job forwards it to `buildEnvelope()`.
5. Rename `eventType` → `operation` and `$eventType` → `$operation` across `CanonicalEventMapper`, `FormatAndPublishSyncEvent`, and both observers.
6. Implement operation-aware `mapChanges()` in `CanonicalEventMapper`: all fields for `create`, `getChanges()` fields for `update` (not `getDirty()` — by the time dispatch runs post-save, `getDirty()` is already cleared), `[]` for `delete`.
7. Delete events must use the full canonical envelope (`operation: "delete"`, `changes: {}`). Remove the custom tombstone format.
8. Extract field definitions into a public `getFieldMap(string $recordType): array` method with an inline note about future `field_metadata` migration.
9. Fix the `startDate` schema declaration from `"format": "date-time"` to `"format": "date"` (date-only, matching the mapper output).
10. At the dispatch call site in the legacy sync service, add a comment documenting that the observer/Redis approach is a shadow-phase simplification and that the [transactional outbox pattern](https://github.com/SuiteDynamics/SuiteX/wiki/Design-%7C-NetSuite-Sync-%E2%80%94-Design,-Requirements,-Deliverables,-and-Runbook#suitex-outbound) is the intended target architecture once shadow mode is replaced by the primary event pipeline.

## Acceptance Criteria

- [ ] `FormatAndPublishSyncEvent` is dispatched exclusively from the legacy NetSuite sync service, after a confirmed `2xx` response — not from Eloquent observers.
- [ ] When the legacy NetSuite call fails (non-2xx, network error, missing `internalId`), no shadow event is queued.
- [ ] Every outbound NetSuite HTTP payload includes `custbody_suitex_write_id` (UUID) and `custbody_suitex_write_source = 'suitex'`.
- [ ] The `writeId` injected into the NetSuite payload is the same value embedded in the corresponding shadow event envelope.
- [ ] Every canonical envelope includes `orderingKey` formatted as `"{recordType}:{recordId}"`.
- [ ] Every canonical envelope includes `actorId` with the authenticated user's ID, or `"system"` when unauthenticated.
- [ ] `actorId` is captured at dispatch time, not resolved inside the queued job.
- [ ] Every canonical envelope includes `baseVersion: null`.
- [ ] Every canonical envelope includes `transactionGroupId: null`.
- [ ] The envelope field is named `operation` (not `eventType`) across all code.
- [ ] `create` events: `changes` contains all mapped fields.
- [ ] `update` events: `changes` contains only the fields returned by `getChanges()` (post-save dirty tracking) — no additional fields included, and `changes` is never empty for a genuine update.
- [ ] `delete` events: full canonical envelope with `operation: "delete"` and `changes: {}`. No custom tombstone.
- [ ] `CanonicalEventMapper` exposes `getFieldMap(string $recordType): array` as a distinct callable method.
- [ ] `startDate` schema declares `"format": "date"`.
- [ ] The dispatch call site in the legacy sync service includes a comment noting the observer/Redis pattern as a shadow-phase simplification and referencing the [transactional outbox pattern](https://github.com/SuiteDynamics/SuiteX/wiki/Design-%7C-NetSuite-Sync-%E2%80%94-Design,-Requirements,-Deliverables,-and-Runbook#suitex-outbound) as the migration target.

## Validation & Testing

1. Unit test `CanonicalEventMapper::buildEnvelope()` for `create` — assert all mapped fields present in `changes`, and all new envelope fields (`orderingKey`, `actorId`, `baseVersion`, `transactionGroupId`) are present.
2. Unit test `buildEnvelope()` for `update` — save the model with two changed fields, then call `buildEnvelope()`. Assert `changes` contains exactly those two fields via `getChanges()` and no others. Confirm `changes` is not empty.
3. Unit test `buildEnvelope()` for `delete` — assert `changes` is empty and `operation` is `"delete"`.
4. Integration test — success path: trigger a Project create via the legacy sync service with a mocked `2xx` NetSuite response. Assert exactly one shadow event is dispatched and its `writeId` matches the value injected into the NetSuite payload.
5. Integration test — failure path: trigger a Project update with a mocked `500` NetSuite response. Assert zero shadow events are dispatched.
6. Run `./vendor/bin/phpstan analyse --level=5 app/Domain/DataSync` — zero errors.
7. Run full `DataSync` domain test suite — zero regressions.

## Related Files (No Changes Required)

- `app/Domain/DataSync/Services/SyncEventPublisherInterface.php` — Defines the publisher contract; not affected by envelope changes.
- `app/Domain/DataSync/Services/LocalLogPublisher.php` — Receives the final envelope; no schema awareness.

## Technical Notes

### Implementation Guidance

The corrected `CanonicalEventMapper` must expose `buildEnvelope()` with the full envelope schema, an operation-aware `mapChanges()` dispatcher, and a public `getFieldMap()` for testability. The dispatch flow must shift entirely out of observers and into the legacy sync service, conditioned on a confirmed `2xx` response.

**Corrected `CanonicalEventMapper`:**

```php
class CanonicalEventMapper
{
    public function __construct(private TenantService $tenantService) {}

    public function buildEnvelope(
        Model $model,
        string $recordType,
        string $operation,
        string $writeId,
        ?string $actorId = null
    ): array {
        return [
            'schemaVersion'      => 'v1',
            'eventId'            => (string) Str::uuid(),
            'accountId'          => $this->tenantService->getCurrentTenantId(),
            'recordType'         => $recordType,
            'recordId'           => (string) $model->id,
            'operation'          => $operation,
            'source'             => 'suitex',
            'timestamp'          => now()->timezone('UTC')->toIso8601String(),
            'orderingKey'        => $recordType . ':' . (string) $model->id,
            'baseVersion'        => null,
            'changes'            => $this->mapChanges($model, $recordType, $operation),
            'fullSnapshotRef'    => null,
            'sourceSystem'       => 'suitex',
            'writeId'            => $writeId,
            'actorId'            => $actorId ?? 'system',
            'transactionGroupId' => null,
        ];
    }

    private function mapChanges(Model $model, string $recordType, string $operation): array
    {
        if ($operation === 'delete') {
            return [];
        }
        if ($operation === 'create') {
            return $this->mapAllFields($model, $recordType);
        }
        return $this->mapDirtyFields($model, $recordType);
    }

    private function mapDirtyFields(Model $model, string $recordType): array
    {
        // getChanges() returns what was modified in the last save — getDirty() is cleared after save()
        // and would always return empty since dispatch runs post-save (after the NetSuite HTTP call).
        $dirty    = $model->getChanges();
        $fieldMap = $this->getFieldMap($recordType);
        $changes  = [];
        foreach ($fieldMap as $dbColumn => $config) {
            if (array_key_exists($dbColumn, $dirty)) {
                $changes[$config['canonical']] = $this->normalizeValue($dirty[$dbColumn], $config['type']);
            }
        }
        return $changes;
    }

    private function mapAllFields(Model $model, string $recordType): array
    {
        $fieldMap = $this->getFieldMap($recordType);
        $changes  = [];
        foreach ($fieldMap as $dbColumn => $config) {
            $changes[$config['canonical']] = $this->normalizeValue($model->getAttribute($dbColumn), $config['type']);
        }
        return $changes;
    }

    public function getFieldMap(string $recordType): array
    {
        // Hardcoded for MVP; migrate to field_metadata lookup in a future epic
        return match ($recordType) {
            'project' => [
                'title'      => ['canonical' => 'title',      'type' => 'string'],
                'isinactive' => ['canonical' => 'isInactive',  'type' => 'boolean'],
                'startdate'  => ['canonical' => 'startDate',   'type' => 'date'],
            ],
            'projecttask' => [
                'title'     => ['canonical' => 'title',     'type' => 'string'],
                'status'    => ['canonical' => 'status',    'type' => 'string'],
                'startdate' => ['canonical' => 'startDate', 'type' => 'date'],
                'enddate'   => ['canonical' => 'endDate',   'type' => 'date'],
            ],
            default => [],
        };
    }

    private function normalizeValue(mixed $value, string $type): mixed
    {
        if ($value === null) {
            return null;
        }
        return match ($type) {
            'boolean' => (bool) $value,
            'date'    => Carbon::parse($value)->toDateString(),
            'string'  => (string) $value,
            default   => $value,
        };
    }
}
```

**Corrected Dispatch Flow:**

```
[User Action]
     │
     ▼
[Legacy NetSuite Sync Service]
     │  1. Generate $writeId = Str::uuid()
     │  2. Inject custbody_suitex_write_id + custbody_suitex_write_source into payload
     │  3. Send HTTP request to NetSuite
     │
     ├─── [NetSuite returns error / no internalId]
     │         └─→ Throw exception. NO shadow event dispatched.
     │
     └─── [NetSuite returns 2xx + internalId]
               │
               ▼
         FormatAndPublishSyncEvent::dispatch(
             model: $model,
             recordType: $recordType,
             operation: $operation,
             writeId: $writeId,
             actorId: auth()->id()  ← captured HERE, not in job
         )
```

> **Note on `actorId` capture timing:** `auth()->id()` returns `null` inside a queued job because the HTTP request context is gone by then. Capture it at dispatch time in the sync service call site and pass it as a constructor argument to the job.

> **Note on `recordId` for `create` shadow events:** The [Legacy API Coexistence spec](https://github.com/SuiteDynamics/SuiteX/wiki/Design-%7C-NetSuite-Sync-%E2%80%94-Design,-Requirements,-Deliverables,-and-Runbook#legacy-api-coexistence--shadow-events-strangler-fig-pattern) states the shadow event "must include the newly minted `recordId`" — referring to the NetSuite `internalId` returned from the API call. If SuiteX models use an auto-increment primary key that differs from the NetSuite internalId, the sync service must pass the returned NetSuite `internalId` to the job as the canonical `recordId` for create events, not `$model->id`. If SuiteX stores the NetSuite internalId directly as the model's primary key, `$model->id` is correct. Confirm the ID strategy before implementing.

**Corrected delete envelope (before/after):**

Before (broken tombstone):
```json
{
  "writeId": "550e8400-e29b-41d4-a716-446655440000",
  "sourceSystem": "suitex",
  "action": "deleted",
  "id": 1234
}
```

After (canonical envelope):
```json
{
  "schemaVersion": "v1",
  "eventId": "7f4e4c3a-1b2d-4e5f-9a0b-123456789abc",
  "accountId": "tenant-uuid",
  "recordType": "project",
  "recordId": "1234",
  "operation": "delete",
  "source": "suitex",
  "timestamp": "2026-03-13T10:00:00+00:00",
  "orderingKey": "project:1234",
  "baseVersion": null,
  "changes": {},
  "fullSnapshotRef": null,
  "sourceSystem": "suitex",
  "writeId": "550e8400-e29b-41d4-a716-446655440000",
  "actorId": "42",
  "transactionGroupId": null
}
```