Platform Integration, Outbox, Worker And Provider Reconciliation — Source Authority

• Status: CURRENT SOURCE-AUTHORITY MAP / DOCUMENTATION ONLY
• Current source baseline: qa/master@a5280b7cdc1c78b809ed6f1996ca31491b522146
• Platform index: PLATFORM_FOUNDATION_RECONSTRUCTION_INDEX.md
• Audit/correlation input: PLATFORM_AUDIT_TRANSITION_CORRELATION_SUPPORT_SOURCE_AUTHORITY.md
• Target: provider-scoped typed integration contracts executed by current PHP adapters or a future separate Node.js worker where explicitly approved
• Phase implementation: USER HOLD
• Effects: no Runtime, DB, queue, provider, credential, callback, file, Wiki DB, deployment or Production mutation

---

1. Authority Decision

CTC has several external-effect and background-work mechanisms. They do not share one current authority and must not be described as one generic queue.

Business Command
= decides and commits the local business result

Effect Intent
= records that a provider/file/notification action is required

Provider Attempt
= one bounded invocation of one provider action

Provider Receipt / Callback
= provider evidence about an attempt or external state

Local Reconciliation
= connects provider evidence back to the owning local process

Scheduled Job
= invokes bounded work according to an external schedule

Worker
= executes provider-specific or long-running work; it is not business authority

Outbox
= durable handoff for one approved effect family; it is not automatically a universal table

1.1 Required invariants

Local DB commit
!= provider accepted

Provider accepted
!= local provider reference saved

HTTP timeout
!= provider failed

Provider error returned
!= local business transaction rolled back

Webhook acknowledged
!= callback applied

Cron endpoint returned 200
!= every eligible item processed

Telegram suppressed
!= Telegram delivered

Gmail Message ID returned
!= every local recipient/part state persisted

QBO SYNCING
!= invoice created
!= invoice absent

File metadata committed
!= object stored
!= object readable
!= every business link complete

Worker picked a job
!= job completed

Dead letter / manual review
!= business cancellation

Generic request log
!= durable provider attempt

1.2 Existing policy boundary

The current accepted policy P01 External Effect Manual Handling Policy explicitly canceled a generic DB outbox/retry/replay design for Gmail appointment mail, Loader Telegram, QBO invoice and QuickBooks Time.

This source authority preserves that decision.

Separate Node.js worker target
!= permission to create one generic outbox

Shared worker runtime
!= shared business/effect authority

Provider-specific durable intent
= allowed only through a new provider-scoped decision and evidence

A future shared worker may provide process hosting, health, leases, tracing, shutdown and metrics. Each provider/effect still owns its typed intent, attempt, result, reconciliation and manual-recovery contract.

---

2. Current Integration Families

Family	Current source owner	Durable local state	Current execution	Current recovery
QBO invoice	BMS AR Billing + QuickBooksOnlineClient	QBO fields on T_TRANSACTION	synchronous browser/API request after local claim	finance/operator review of local state and QBO; no general provider search client
QuickBooks Time	Staff Time + QuickBooks Time helper/client	T_STAFF_TIME_EVENT, user mapping, provider/config state and JSONL	synchronous post-commit sync from selected workforce actions	provider-specific conflict/manual-review result and setup Runbook
Gmail outgoing	caller + GmailSender	generally caller-owned business fields; Gmail Message/Thread ID only where caller stores it	synchronous Gmail API send	caller-specific reread/manual resend; no universal send-attempt row
Gmail appointment intake	Gmail Watch + Intake Service + Appointment Store	T_MAILBOX_WATCH_STATE, T_MAIL_MESSAGE, T_MAIL_REQUEST_ITEM	Pub/Sub callback, history sync and manual replay	history-cursor replay and operator approval queue
Telegram outgoing	caller + TelegramSender	caller-specific planner/break fields where present; no common attempt row	synchronous Bot API request	caller-specific/manual notification
Telegram webhook/cron	public secret-protected endpoints	business rows modified by invoked owner; no webhook receipt ledger identified	direct webhook/callback or HTTP cron execution	request response/log and owner readback
AI Voice / Twilio	TMS AI Voice + Twilio callbacks	T_AI_VOICE_CASE, T_AI_VOICE_CALL, transcript segments, provider/recording fields	local intent first, synchronous Twilio start, callbacks, separate voice gateway	explicit known failure/provider unknown/manual follow-up and recording cleanup retry
Voice transcript outbox	Node voice gateway	filesystem JSON entries	periodic drain plus immediate three-attempt delivery	retain file for later drain until expiry; invalid/expired entries removed/logged
Gemini live voice	Node voice gateway	call/session state in CTC plus gateway memory	long-lived WebSocket and signed CTC tool calls	bounded fallback to Office and call/provider state
File object/storage	FileStorage + shared file owner	T_FILE, object in EFS/S3, business link fields	synchronous object I/O and DB registration/linking	compensation, pending object cleanup, archive retry
Root cron/mail jobs	cron.php and external scheduler	business marker timestamps plus JSONL	synchronous CLI run	rerun/manual review by job-specific state; no durable job-run authority identified
Telegram break cron	HTTP cron endpoint	Break/Staff Time/Planner rows	secret-protected HTTP invocation	owner readback; no durable lease/run ledger identified
AI Voice retention	retention script/systemd timer	call/recording/transcript fields	bounded scheduled deletion and provider cleanup	DELETE_FAILED state and retry counters

---

3. QBO Invoice Current Authority

3.1 Current request path

POST /api/bms/ar-billing/qbo-sync
-> validate AR billing group
-> conditionally claim rows as SYNCING
-> place random claim token in QBO_ERROR
-> leave local DB transaction
-> QuickBooksOnlineClient.createInvoice()
-> persist QBO Invoice ID / Doc Number / Sync Token
-> set SYNCED

The current QBO client exposes invoice creation only. It does not expose a source-confirmed invoice search/reconciliation operation.

3.2 Current local fields

QBO_INVOICE_ID
QBO_DOC_NUMBER
QBO_SYNC_TOKEN
QBO_SYNC_STATUS
QBO_SYNCED_AT
QBO_ERROR

Observed status family includes:

READY
SYNCING
SYNCED
ERROR

The claim token is temporarily multiplexed through QBO_ERROR. It is not an immutable provider-attempt ID.

3.3 Critical uncertainty boundary

QBO create accepted
-> response received
-> local result save fails

or

QBO create accepted
-> network response lost

The local row can become ERROR, remain SYNCING or otherwise lack the provider reference even though a QBO invoice exists. Blind replay can create a duplicate invoice.

3.4 Current limitations

• no durable typed attempt history;
• no source-confirmed provider request identity;
• no provider search/reconcile method in the inspected QBO client;
• no accepted stale-SYNCING timeout/owner policy;
• claim state and error text share one column;
• retry behavior depends on current finance workflow and operator judgment;
• QBO credentials and provider acceptance were not tested in this documentation wave.

Existing detailed business ownership remains in the Billing reconstruction packet. This platform document only defines shared integration boundaries.

---

4. QuickBooks Time Current Authority

4.1 Local authority

T_STAFF_TIME_EVENT
= local workforce event authority

QuickBooks Time is a downstream provider projection. Provider success or failure does not rewrite the meaning of the local staff event.

4.2 Current execution

The helper:

• resolves tenant/module/config;
• refreshes OAuth tokens synchronously when required;
• writes refreshed token values through tenant config;
• maps a local user to QUICKBOOKS_TIME_USER_ID;
• reads active provider timesheets;
• creates/updates/closes provider timesheets according to the event;
• logs provider-specific results to JSONL;
• returns manual_review_required for external or conflicting active timesheets;
• can return unknown when provider write results are unreadable.

4.3 Current limits

Local staff event committed
!= QuickBooks Time synchronized

QuickBooks Time active timesheet exists
!= safe to overwrite

No generic durable attempt ledger was identified. Current recovery remains provider-specific and manual when conflicts or ambiguous results exist.

Token refresh is itself a provider/config effect and can fail independently from the timesheet operation.

---

5. Gmail Outgoing Current Authority

5.1 Sender behavior

GmailSender builds MIME content and synchronously calls Gmail users.messages.send.

A successful result can return:

Gmail Message ID
Gmail Thread ID
Label IDs

An exception is converted to a Code=N result.

5.2 Attachment behavior

An attachment whose local path is absent is skipped by the sender. Therefore:

Gmail send succeeded
!= every intended attachment was included

The caller must validate the required attachment set before the send. The provider adapter must not silently decide which business evidence is optional.

5.3 Current caller patterns

• BOL and Packing List routes send synchronously and return provider result;
• Appointment reply workflows store Gmail reply IDs in request-item rows;
• root cron sends late/no-show/ASN/report mail and updates business marker timestamps only after reported send success;
• other direct callers may rely primarily on JSONL/log response.

5.4 Critical uncertainty boundary

Gmail accepted message
-> local marker/update/reply-ID persistence fails

A later job or user can resend because the local owner still appears pending.

No universal outgoing-mail attempt table was identified. Existing T_MAIL_MESSAGE / T_MAIL_REQUEST_ITEM are appointment-intake and workflow records, not automatically the authority for every outgoing email.

---

6. Gmail Appointment Watch And Intake Authority

This family has a more durable current model than generic outgoing mail.

6.1 Watch state

T_MAILBOX_WATCH_STATE owns:

mailbox
queue label
Pub/Sub topic
last history ID
watch expiration
last sync time
last notification time
status
last error

6.2 Pub/Sub handling

validate webhook token
-> decode mailbox/history notification
-> reject mailbox mismatch
-> load watch state
-> duplicate/stale history -> noop
-> history gap / missing baseline -> needs replay
-> sync Gmail history
-> ingest changed messages
-> advance history cursor

6.3 Intake state

T_MAIL_MESSAGE
= provider message/thread and extracted mail envelope

T_MAIL_REQUEST_ITEM
= one operator-review work item derived from the message

The appointment store persists a message and immutable item batch in one local transaction. Request items carry action type, current status, matched load, proposed/resolved time, action owner/note, reply IDs and error state.

6.4 Important distinction

Pub/Sub notification received
!= message ingested

Message ingested
!= request approved

Request approved
!= appointment mutation and reply both complete

This family is a provider/domain-specific durable workflow. It must not be generalized into a universal outbox without an explicit design decision.

---

7. Telegram Current Authority

7.1 Outbound sender

TelegramSender:

• resolves tenant module/config and token;
• sends directly to the Telegram Bot API;
• uses short connect/request timeouts;
• returns decoded provider response;
• does not write a common provider-attempt row.

When delivery is disabled it returns:

{
  "ok": true,
  "suppressed": true
}

Therefore:

ok=true + suppressed=true
!= message delivered

Every caller and future UI must display SUPPRESSED separately from SENT.

7.2 Webhook

The Telegram webhook:

• validates a tenant/provider secret;
• accepts message or callback input;
• resolves the chat to exactly one tenant staff user for relevant actions;
• handles selected callback/edit/fallback sends;
• currently redirects Staff Time actions to the Loader page;
• can return HTTP/application success while including an internal handled error and fallback response.

No durable Telegram Update ID receipt/deduplication store was identified in the inspected source.

7.3 Cron endpoint

The Telegram cron endpoint:

• is an HTTP endpoint;
• accepts a tenant target and secret;
• invokes expired Break closure;
• returns a summary result.

No durable job-run ID, lease, heartbeat or replay ledger was identified.

---

8. AI Voice / Twilio / Gemini Current Authority

This is the most complete current provider-specific intent/result model.

8.1 Local call intent

Outbound start:

validate Office / tenant / storage / provider config
-> create or reuse local AI Voice Call by REQUEST_KEY
-> commit local intent
-> if newly created, call Twilio
-> persist known provider result
-> mark known failure or PROVIDER_UNKNOWN

When the provider outcome is unknown, the API explicitly tells the operator not to retry the Load.

8.2 Durable current records

T_AI_VOICE_CASE
T_AI_VOICE_CALL
T_AI_VOICE_TRANSCRIPT_SEGMENT

Call state includes provider SID, direction, purpose, status, session state, matching and recording fields. Recording deletion has status, retry count and error fields.

8.3 Callback authority

Twilio status and recording callbacks:

• validate the provider request;
• apply provider state in a local DB transaction;
• return TwiML/HTTP response;
• can be retried by Twilio on failure.

Callback receipt and business application are currently combined in the endpoint transaction; a separate callback-receipt ledger was not identified.

8.4 Voice gateway

The Node gateway owns:

• Twilio Media Stream WebSocket;
• Gemini Live WebSocket;
• bounded signed CTC tool calls;
• live-session activity/no-response handling;
• transcript sequencing and delivery.

Gateway tool requests use HMAC, timestamp and nonce. The CTC side conditionally stores nonces on the call row to reject replay.

8.5 Transcript filesystem outbox

The voice gateway implements a real provider-specific outbox:

transcript segment
-> secure JSON file staged atomically
-> immediate delivery with three attempts
-> leave file if not delivered
-> periodic 30-second drain, bounded to 100 files
-> delete after CTC accepts
-> delete/log invalid or expired entries

This outbox is:

• transcript-specific;
• filesystem-based;
• local to the gateway host/storage;
• not a universal DB queue;
• dependent on one process owning the directory and drain behavior;
• potentially unsuitable for multi-instance deployment without shared ownership/locking decisions.

8.6 Retention cleanup

AI Voice retention performs provider recording deletion, retry tracking, transcript/call/case deletion and selected Operation Event cleanup. It is a provider-specific destructive maintenance workflow and must remain separately authorized from ordinary call processing.

8.7 Current structural compatibility

The inspected source includes a numeric Client 1 restriction for AI Voice. This is current compatibility behavior, not the target capability/profile model.

---

9. File Object And Cleanup Authority

File handling contains independent checkpoints.

FILE_OBJECT
-> EFS/S3 bytes

FILE_METADATA
-> T_FILE row

FILE_LINK
-> TABLE_NAME / TABLE_ID / TYPE relationship

FILE_READABILITY
-> object can be opened/read

OBJECT_CLEANUP
-> physical deletion and cleanup marker

9.1 Upload order

A common path is:

store object
-> register metadata/link in DB
-> on DB failure, attempt object compensation

Compensation failure can leave an orphan object.

9.2 Delete order

A Wiki asset path can:

soft-delete metadata/link
-> delete object
-> set OBJECT_CLEANUP_AT

Object deletion failure is logged and leaves cleanup pending.

9.3 Archive cleanup

The archive workflow contains bounded retry behavior for pending file cleanup and preserves object/link/archive distinctions. It is a maintenance job, not a generic provider worker.

---

10. Current Scheduler And Worker Reality

The current repository contains several execution styles.

Style	Example	Current limitation
Browser/API synchronous provider call	QBO invoice, BOL/Packing Gmail, Telegram, Twilio start	request timeout and provider/local partial result
CLI cron script	root mail cron, archive, retention	external scheduler ownership and limited durable run history
Secret-protected HTTP cron	Telegram Break cleanup	request result only; no lease/run ledger identified
Provider webhook	Telegram, Twilio, Gmail Pub/Sub	duplicate/out-of-order/security/retry semantics differ by provider
Persistent Node service	AI Voice Gateway	process/service supervision and local filesystem ownership required
Filesystem outbox	voice transcript	host-local durability and single-drainer assumptions
Domain work-item tables	Gmail appointment intake, AI Voice call	provider/domain-specific durable state

There is no current universal Node worker, generic outbox table, universal attempt table or universal dead-letter queue established by the inspected source.

---

11. Current Credential Ownership

Provider	Current credential source/owner
QBO invoice	tenant config/env used by billing helper/client
QuickBooks Time	tenant config/env, including refresh/access token updates
Gmail	credential JSON path plus tenant-scoped token file, with legacy token-path fallback
Telegram	tenant provider config/env token and webhook/cron secrets
Twilio	tenant config/env account SID/auth token/numbers/URLs
Gemini	voice gateway config, with call-specific safe delivery from CTC gateway binding
S3	deployment environment/config credentials

Credential availability does not imply effect authorization. Provider clients must receive a credential-owner reference matching the effective tenant/deployment policy.

No secret value belongs in support timelines, operation events, transition audit, Wiki, error payloads or provider-attempt display.

---

12. Current Correlation Inputs

Existing families expose some of:

requestId
business request key/token
provider message/thread/call/invoice/recording ID
mail history ID
Pub/Sub message ID
call UUID
transcript sequence
Telegram callback query/message/chat ID
file UUID/path
business record ID/UUID

These identifiers are not normalized into one current provider-attempt contract. The Audit/Correlation packet defines the target relationship:

requestId
-> commandRequestToken
-> processInstanceId / correlationId
-> auditId
-> effectIntentId
-> providerAttemptId
-> providerReceiptId / callbackReceiptId

A future provider worker must reuse this chain rather than inventing a second correlation authority.

---

13. Current Evidence Ceiling And Unknowns

The following were not verified in this documentation wave:

• Production provider credentials or connectivity;
• live QBO invoices, QBO Time timesheets, Gmail messages, Telegram messages or Twilio calls;
• exact DB schema/rows for every provider family;
• external scheduler configuration and last-run state;
• systemd service/timer state for the voice gateway/retention;
• current filesystem outbox contents or permissions;
• multi-instance worker behavior;
• current S3/EFS objects;
• webhook delivery/retry history;
• provider rate limits and quotas;
• current dead-letter/manual-review volume;
• legal/contractual retention requirements;
• target-role operational acceptance.

These remain RUNTIME / PROVIDER / DEPLOYMENT EVIDENCE REQUIRED.

---

14. Prohibited Interpretation

Do not claim:

• CTC already has a universal outbox/worker;
• the voice transcript filesystem outbox is suitable for all providers;
• the accepted manual-handling policy has been superseded;
• SYNCING, ERROR, HTTP 200 or a JSONL line proves provider state;
• Telegram suppressed=true is delivery;
• Gmail send success proves attachment completeness or local persistence;
• Pub/Sub/Webhook acknowledgement proves business completion;
• QBO invoice replay is safe without provider reconciliation;
• all current callbacks are idempotent or ordered;
• a Node worker may own business state transitions;
• this documentation authorizes a provider call, schema change, worker deployment or retry activation.

---

15. Source Inventory

QBO / QuickBooks Time

• app/utils/QuickBooksOnlineClient.php
• app/systems/BMS/ar-billing/api_routes.php
• app/systems/BMS/ar-billing/ar_billing_helper.php
• app/systems/TMS/staff-time/quickbooks_time_sync_helper.php
• app/addons/quickbooks/*
• public/quickbooks-time/*

Gmail / Mail intake

• app/utils/GmailClientFactory.php
• app/utils/GmailSender.php
• app/utils/GmailAppointmentWatchService.php
• app/utils/TransloadAppointmentIntakeService.php
• app/utils/TransloadAppointmentStore.php
• app/utils/GmailAppointmentAddonGateway.php
• public/gmail/*
• cron.php

Telegram

• app/utils/TelegramSender.php
• app/addons/telegram/api_routes.php
• public/telegram/webhook.php
• public/telegram/cron.php
• Loader/Planning/Break callers

AI Voice / Twilio / Gemini

• app/systems/TMS/ai-voice/api_routes.php
• app/systems/TMS/ai-voice/voice_helper.php
• app/systems/TMS/ai-voice/twilio_voice_client.php
• public/integrations/twilio/*
• voice-gateway/src/*
• voice-gateway/deploy/*
• scripts/ai_voice_retention.php

Files / Maintenance

• app/shared/files/api_routes.php
• app/shared/files/media_helper.php
• app/shared/storage/file_storage.php
• scripts/archive_prod_data.php
• docs/runbooks/DATA_ARCHIVE_RUNBOOK.md

Policy / Existing packets

• docs/reference/modernization-v1/contracts/P01-external-effect-handling-policy.md
• docs/reconstruction/CTC_BILLING_QBO_FILES_EMAIL_CAPABILITY_REBUILD_PACKET.md
• docs/reconstruction/PLATFORM_AUDIT_TRANSITION_CORRELATION_SUPPORT_REBUILD_PACKET.md

---

16. Downstream Documentation Owner

The target model, typed contracts, worker boundaries, retry/reconciliation rules and migration sequence are defined in:

• PLATFORM_INTEGRATION_OUTBOX_WORKER_PROVIDER_REBUILD_PACKET.md

This source authority must be updated when a provider caller, durable state, callback, credential owner, retry rule, scheduler or worker deployment changes.

---

Source: docs/reconstruction/PLATFORM_INTEGRATION_OUTBOX_WORKER_PROVIDER_SOURCE_AUTHORITY.md — ctc repository · This page is an English translation of the source document. Identifiers, file paths and code symbols are unchanged.