LOE: 5 (M) | Rationale: Six targeted extensions to the existing Epic 3 Node.js Mock Server (NestJS/Cloud Run). The mock server codebase, deployment pipeline, and persistence layer already exist — this work adds new endpoints and behaviors on top of that foundation. The extensions are straightforward CRUD/simulation endpoints with no complex business logic, but require careful contract alignment with Epic 7's consumers and accurate simulation of NetSuite API behaviors (versioning, marker fields, error responses). The UE/polling event simulation endpoints require understanding the canonical `ChangeEvent` envelope schema from Epic 1.

# Jira Task: Epic 7.2 — Mock Server Extensions for Merge Orchestrator Development

**Title:** Extend Epic 3 Mock Server to Support Epic 7 Merge Orchestrator Development & Testing
**Priority:** High
**Story Points:** 5 (M)
**Type:** Feature
**Assignee:** TBD
**Parent Epic:** Epic 7 — Merge Orchestrator & Three-Way Sync Engine
**Labels:** `data-sync`, `epic-7`, `epic-3`, `mock-server`, `test-infrastructure`, `netsuite-simulation`
**Blocks:** [Epic 7 — Merge Orchestrator (Stage 3+)](jira-task-gap-resolution.md)
**Depends On:** Epic 1 (Canonical Schema — for `ChangeEvent` envelope format), Epic 3 (Mock Server — base codebase)

---

## Context

Epic 7 builds the Merge Orchestrator — five stateless consumers that process raw change events, perform three-way merges, resolve conflicts, suppress circular updates, and deliver changes to SuiteX and NetSuite. All development and integration testing must target the Epic 3 Node.js Mock Server rather than live NetSuite to avoid exhausting governance limits and to enable deterministic testing of error paths.

The Epic 3 mock server's current scope covers:
- **RESTlet write endpoints** (`POST /mock-netsuite/restlet/project`, `POST /mock-netsuite/restlet/project-task`) — accepts and validates Epic 1 canonical JSON payloads
- **Cursor-based poller/search endpoint** — accepts cursor queries, returns paginated results (1,000 limit)
- **Chaos middleware** — configurable latency (200ms–2000ms) and 429 responses at 5 concurrent connections
- **State persistence** — Cloud SQL (MySQL 8.*) with auto-generated `internalId` and `lastModifiedDate`

This scope does not cover several interactions required by Epic 7's consumers. Without these extensions, Epic 7 cannot begin Stage 3 (Consumers) or Stage 5 (Integration Tests).

This ticket is the authoritative source for all mock server extension requirements. The [Epic 7 main task](jira-task-gap-resolution.md) references this ticket but does not duplicate the technical specifications.

---

## Description

Extend the Epic 3 Node.js Mock Server with six new capabilities that enable full development and integration testing of the Epic 7 Merge Orchestrator. The extensions fall into three categories:

**API Surface Extensions (2):**
1. Record Read endpoint for the RemoteCurrent fetch pattern
2. Marker field storage and retrieval on existing endpoints

**Event Simulation (2):**
3. UE event generation and publication to `events.raw`
4. Polling event generation with matching fingerprints

**Test Instrumentation (2):**
5. Concurrent modification injection
6. Per-error-class error injection

### Areas to Review

- [Epic 3 design-spec](../epic3/design-spec.md) — Current mock server scope and architecture
- [Epic 7 design-spec](design-spec.md) — RemoteCurrent fetch mechanism, circular update prevention, UE/polling deduplication
- [Epic 1 tech-spec](../epic1/tech-spec.md) — Canonical `ChangeEvent` envelope schema (required for event simulation)
- [Appendix F2 — NetSuite Writer pseudocode](https://github.com/SuiteDynamics/SuiteX/wiki/Design-%7C-NetSuite-Sync-%E2%80%94-Design,-Requirements,-Deliverables,-and-Runbook#f2-netsuite-writer) — Read-verify-update pattern

---

## Deliverables

### 1. Record Read Endpoint (RemoteCurrent)

**Endpoint:** `GET /mock-netsuite/restlet/{recordType}/{recordId}`

**Purpose:** The NetSuite Writer performs a read-verify-update pattern — it fetches the target record's current state before writing. This endpoint simulates that live GET read.

**Behavior:**
- Returns the record's current field values as a JSON object
- Includes `lastModifiedDate` (ISO8601 UTC) and a monotonic `version` number (integer, incremented on every write)
- Accepts an optional `fields` query parameter (comma-separated list of field names) — if provided, only returns those fields; if omitted, returns all fields. This allows tests to verify that the Writer only fetches synced fields.
- Returns `404` if the record does not exist
- Respects the existing chaos middleware (latency, 429s)

**Response schema:**
```json
{
  "internalId": "123",
  "recordType": "project",
  "version": 5,
  "lastModifiedDate": "2026-03-13T10:00:00.000Z",
  "fields": {
    "companyname": "Acme Corp",
    "status": "Active",
    "custbody_suitex_write_id": "uuid-here",
    "custbody_suitex_write_source": "suitex"
  }
}
```

### 2. Marker Field Storage & Retrieval

**Extends:** Existing `POST /mock-netsuite/restlet/{recordType}` write endpoints and the search/poller endpoint.

**Purpose:** SuiteX writes include marker fields (`custbody_suitex_write_id`, `custbody_suitex_write_source`) that NetSuite UE scripts inspect to suppress circular re-emission. The mock server must persist these so circular update prevention and polling suppression can be tested end-to-end.

**Behavior:**
- Write endpoints accept `custbody_suitex_write_id` (string/UUID) and `custbody_suitex_write_source` (string, typically `"suitex"`) in the request payload
- These fields are persisted alongside the record in the mock database
- The poller/search endpoint returns these fields in its response for each record
- The Record Read endpoint (Deliverable 1) also returns these fields
- Marker fields are cleared when explicitly set to `null` in a subsequent write (simulates the polling cleanup cycle)

### 3. UE Event Simulation

**Endpoint:** `POST /mock-netsuite/simulate/ue-event`

**Purpose:** Generates canonical `ChangeEvent` payloads simulating NetSuite User Event script emissions and publishes them to the `events.raw` Pub/Sub topic. This allows the Merge Service to be tested with realistic NetSuite-originated events without deploying actual UE scripts, decoupling Epic 7 from Epic 5.

**Request schema:**
```json
{
  "accountId": "acct_123",
  "recordType": "project",
  "recordId": "456",
  "operation": "update",
  "changes": {
    "status": "Closed",
    "companyname": "Acme Corp Updated"
  },
  "sourceSystem": "user",
  "actorId": "ns_user_789",
  "baseVersion": 5
}
```

**Behavior:**
- Constructs a full `ChangeEvent` envelope conforming to the Epic 1 canonical schema (v1)
- Auto-generates: `eventId` (UUID), `writeId` (UUID), `timestamp` (current UTC), `orderingKey` (`{recordType}:{recordId}`), `schemaVersion` ("v1")
- Sets `source: "netsuite-ue"`
- Publishes the event to the `events.raw` Pub/Sub topic (or local emulator)
- Also mutates the record in the mock database to reflect the changes (so a subsequent RemoteCurrent GET returns the updated state)
- Increments the record's `version` in the mock database
- Returns the full constructed `ChangeEvent` payload in the response for test assertions

### 4. Polling Event Simulation

**Endpoint:** `POST /mock-netsuite/simulate/poll-event`

**Purpose:** Generates polling-style events with `source: "netsuite-poll"` for the same record. When the changes match a prior UE event, the fingerprints will match, allowing UE/polling deduplication testing.

**Request schema:** Same as UE event simulation.

**Behavior:**
- Same as UE event simulation except `source` is set to `"netsuite-poll"`
- The `changes` and record identifiers should match a prior UE event to produce matching fingerprints for deduplication testing
- Does NOT mutate the mock database (polling is a read-only observation, not a write)
- Returns the full constructed `ChangeEvent` payload in the response

### 5. Concurrent Modification Simulation

**Endpoint:** `POST /mock-netsuite/admin/inject-mutation`

**Purpose:** Simulates the scenario where another actor modifies a record between the Writer's GET (RemoteCurrent fetch) and PUT (write), causing a version mismatch. This tests the `ConcurrentModification` retry path.

**Request schema:**
```json
{
  "recordType": "project",
  "recordId": "456",
  "changes": {
    "status": "Cancelled"
  },
  "triggerOn": "next-write"
}
```

**Behavior:**
- When `triggerOn: "next-write"`: the next PUT/POST to this record will be preceded by an automatic internal mutation that increments the record's `version` and applies the specified `changes`. The PUT then fails with a `409 Conflict` (or equivalent) because the version no longer matches what the Writer saw during its GET.
- When `triggerOn: "immediate"`: the mutation is applied immediately (useful for setting up state before a test sequence).
- The injection is consumed after one trigger (single-use) — subsequent writes proceed normally.
- Returns `200` with `{ "injected": true, "recordType": "project", "recordId": "456", "triggerOn": "next-write" }`

### 6. Error Injection (Per Error Class)

**Endpoint:** `POST /mock-netsuite/admin/inject-error`

**Purpose:** Allows targeted error injection per record or per request type so all six error classes in Epic 7's retry policy can be tested deterministically, beyond the existing chaos middleware's latency and 429 capabilities.

**Request schema:**
```json
{
  "recordType": "project",
  "recordId": "456",
  "errorClass": "validation",
  "httpStatus": 400,
  "errorBody": {
    "code": "INVALID_FLD_VALUE",
    "message": "Invalid value for field 'status'"
  },
  "triggerCount": 3,
  "applyTo": "write"
}
```

**Behavior:**
- `errorClass` must be one of: `transient`, `rate-limit`, `governance`, `validation`, `auth`, `conflict`
- `httpStatus` defaults per error class if not specified: `transient` → 503, `rate-limit` → 429, `governance` → 429, `validation` → 400, `auth` → 401, `conflict` → 409
- `triggerCount` specifies how many requests will receive this error before normal behavior resumes. Set to `-1` for indefinite (until cleared).
- `applyTo` specifies which request types are affected: `"read"`, `"write"`, `"both"`. Default: `"write"`.
- When `recordId` is `"*"`, the error applies to all records of that type.
- Active injections can be listed via `GET /mock-netsuite/admin/inject-error` and cleared via `DELETE /mock-netsuite/admin/inject-error/{injectionId}`.
- Returns `200` with the injection ID and details.

**Error class → HTTP response mapping:**

| Error Class | HTTP Status | Response Body Pattern |
|---|---|---|
| `transient` | 503 | `{ "error": { "code": "SSS_REQUEST_LIMIT_EXCEEDED" } }` |
| `rate-limit` | 429 | `{ "error": { "code": "REQUEST_LIMIT_EXCEEDED" } }` |
| `governance` | 429 | `{ "error": { "code": "SSS_USAGE_LIMIT_EXCEEDED" } }` |
| `validation` | 400 | `{ "error": { "code": "INVALID_FLD_VALUE", "message": "..." } }` |
| `auth` | 401 | `{ "error": { "code": "INVALID_LOGIN_CREDENTIALS" } }` |
| `conflict` | 409 | `{ "error": { "code": "RCRD_HAS_BEEN_CHANGED", "message": "Record has been changed" } }` |

---

## Acceptance Criteria

**Record Read (RemoteCurrent)**

- [ ] `GET /mock-netsuite/restlet/project/123` returns the record's current field values, `version`, and `lastModifiedDate`
- [ ] `GET /mock-netsuite/restlet/project/123?fields=status,companyname` returns only the requested fields
- [ ] `GET /mock-netsuite/restlet/project/999` returns 404 for non-existent records
- [ ] Version increments monotonically on every write to the record
- [ ] Chaos middleware (latency, 429s) applies to read requests

**Marker Field Storage**

- [ ] `POST /mock-netsuite/restlet/project` with `custbody_suitex_write_id` and `custbody_suitex_write_source` in the payload persists both fields
- [ ] Subsequent `GET /mock-netsuite/restlet/project/{id}` returns the stored marker fields
- [ ] Poller/search endpoint returns marker fields in its response
- [ ] Setting marker fields to `null` in a subsequent write clears them

**UE Event Simulation**

- [ ] `POST /mock-netsuite/simulate/ue-event` constructs a valid `ChangeEvent` envelope conforming to Epic 1 canonical schema v1
- [ ] Event has `source: "netsuite-ue"` and auto-generated `eventId`, `writeId`, `timestamp`, `orderingKey`
- [ ] Event is published to `events.raw` Pub/Sub topic
- [ ] The mock database is mutated to reflect the changes (RemoteCurrent GET returns updated state)
- [ ] Record `version` is incremented in the mock database
- [ ] Response includes the full constructed `ChangeEvent` payload

**Polling Event Simulation**

- [ ] `POST /mock-netsuite/simulate/poll-event` constructs a valid `ChangeEvent` with `source: "netsuite-poll"`
- [ ] When given the same `changes` as a prior UE event for the same record, the fingerprint matches
- [ ] Mock database is NOT mutated (polling is read-only observation)
- [ ] Response includes the full constructed `ChangeEvent` payload

**Concurrent Modification**

- [ ] `POST /mock-netsuite/admin/inject-mutation` with `triggerOn: "next-write"` causes the next write to that record to fail with a version mismatch
- [ ] The injection is consumed after one trigger — subsequent writes succeed
- [ ] `triggerOn: "immediate"` mutates the record immediately
- [ ] The version increment and field changes are visible via the Record Read endpoint

**Error Injection**

- [ ] Each of the six error classes produces the correct HTTP status code and response body pattern
- [ ] `triggerCount` controls how many requests receive the error; after exhaustion, normal behavior resumes
- [ ] `triggerCount: -1` produces indefinite errors until explicitly cleared
- [ ] `recordId: "*"` applies the error to all records of that type
- [ ] Active injections are listable via `GET /mock-netsuite/admin/inject-error`
- [ ] Individual injections are clearable via `DELETE /mock-netsuite/admin/inject-error/{injectionId}`
- [ ] `applyTo: "read"` only affects GET requests; `applyTo: "write"` only affects POST/PUT

**Integration with Epic 7**

- [ ] `NetSuiteWriter` successfully performs read-verify-update against the mock server's Record Read and Write endpoints
- [ ] Merge Service successfully consumes UE-simulated events from `events.raw`
- [ ] UE/polling deduplication works with simulated events (matching fingerprints)
- [ ] Circular update prevention works end-to-end with marker fields persisted and returned by the mock
- [ ] All six error classes in the retry policy are testable via error injection
- [ ] The `ConcurrentModification` retry path is exercised via concurrent modification injection

---

## Validation & Testing

1. **RemoteCurrent round-trip:** Write a record via POST → read it back via GET → confirm field values, version, and `lastModifiedDate` match. Write again → confirm version incremented.
2. **Field filtering:** Write a record with 10 fields → GET with `fields=status,companyname` → confirm only 2 fields returned.
3. **Marker field lifecycle:** Write with marker fields → GET → confirm markers present → write with markers set to `null` → GET → confirm markers absent.
4. **UE event validity:** Simulate a UE event → capture the published message from `events.raw` → validate against Epic 1 canonical schema using a draft-07 JSON Schema validator.
5. **UE/polling fingerprint match:** Simulate a UE event with specific `changes` → simulate a poll event with identical `changes` for the same record → compute `hash(recordType, recordId, lastModifiedDate, operation)` for both → confirm fingerprints match.
6. **Concurrent modification flow:** GET a record (note version=5) → inject mutation with `triggerOn: "next-write"` → attempt PUT → receive 409 → GET again (note version=6 with injected changes) → retry PUT with updated version → success.
7. **Error injection exhaustion:** Inject a `transient` error with `triggerCount: 3` → send 3 write requests → confirm all return 503 → send 4th request → confirm success (200).
8. **Error injection wildcard:** Inject `auth` error with `recordId: "*"` → write to any record of that type → confirm 401 → clear injection → write again → confirm success.
9. **End-to-end with Epic 7 consumers:** Simulate a UE event via mock → Epic 7 Merge Service consumes and validates → NetSuite Writer fetches RemoteCurrent from mock → three-way merge executes → Writer writes to mock with marker fields → simulate a polling event for the same record → verify deduplication. All with zero live NetSuite calls.

---

## Related Files

- [Epic 7 main task](jira-task-gap-resolution.md) — The Merge Orchestrator implementation task that depends on these mock server extensions; defines the consumer behaviors that these extensions must support
- [Epic 7 design-spec](design-spec.md) — Architecture, RemoteCurrent fetch mechanism, circular update prevention, UE/polling deduplication requirements
- [Epic 3 design-spec](../epic3/design-spec.md) — Current mock server scope: RESTlet write endpoints, poller, chaos middleware, Cloud Run/Cloud SQL deployment
- [Epic 1 tech-spec](../epic1/tech-spec.md) — Canonical `ChangeEvent` envelope schema (v1) — required for UE/polling event simulation
- [Design document — F2 pseudocode](https://github.com/SuiteDynamics/SuiteX/wiki/Design-%7C-NetSuite-Sync-%E2%80%94-Design,-Requirements,-Deliverables,-and-Runbook#f2-netsuite-writer) — NetSuite Writer read-verify-update pattern that the Record Read endpoint must support

---

## Technical Notes

### Mock Database Schema Extensions

The existing `MockRecord` entity in the Epic 3 database must be extended with:

```sql
ALTER TABLE mock_records
    ADD COLUMN version BIGINT UNSIGNED NOT NULL DEFAULT 1,
    ADD COLUMN custbody_suitex_write_id VARCHAR(36) NULL,
    ADD COLUMN custbody_suitex_write_source VARCHAR(50) NULL;
```

The `version` column must be incremented on every `UPDATE` to the record. The `lastModifiedDate` column (already present) must also be updated on every write.

### Error Injection State

Error injections are stored in-memory (not persisted to the database) since they are transient test instrumentation. The data structure:

```typescript
interface ErrorInjection {
  id: string;                // Auto-generated UUID
  recordType: string;
  recordId: string;          // "*" for wildcard
  errorClass: string;
  httpStatus: number;
  errorBody: object;
  triggerCount: number;      // -1 for indefinite
  remainingTriggers: number;
  applyTo: 'read' | 'write' | 'both';
}
```

The injection middleware checks for matching injections before processing any request. When `remainingTriggers` reaches 0, the injection is automatically removed.

### Concurrent Modification State

Similar to error injections, concurrent modification triggers are stored in-memory:

```typescript
interface MutationInjection {
  recordType: string;
  recordId: string;
  changes: Record<string, any>;
  triggerOn: 'next-write' | 'immediate';
  consumed: boolean;
}
```

When `triggerOn: "next-write"`, the write endpoint handler checks for a pending injection before processing the write. If found, it first applies the injected mutation (incrementing `version`), then attempts to process the incoming write — which will fail because the version no longer matches.

### Pub/Sub Integration

The UE and polling event simulation endpoints publish to the `events.raw` Pub/Sub topic. In local development, this targets the Pub/Sub emulator. The endpoint must:
1. Construct the full `ChangeEvent` envelope
2. Set the Pub/Sub message's `orderingKey` attribute to `{recordType}:{recordId}`
3. Publish the message
4. Return the constructed payload in the HTTP response

### Deployment

These extensions are deployed as part of the existing Epic 3 Cloud Run container. No new infrastructure is required. The admin and simulate endpoints should be protected by an API key or network-level restriction (Cloud Run IAM) to prevent accidental use in non-test environments.