Downstream HTI CRM Delivery & Retry Queue

Once qualified leads are accepted through public forms or partner webhooks, they are forwarded in real time to the agency's primary operational CRM (an enterprise Falcon/Zoho-backed management platform).

The delivery pipeline is engineered to be asynchronous, non-blocking, and fail-safe, guaranteeing that temporary downstream CRM outages never degrade customer-facing website performance.

---

1. Fire-and-Forget Forwarding Architecture

The client user interface is never blocked waiting for external CRM roundtrips. Lead delivery executes asynchronously via sendLeadToHtiCrm() in src/lib/hti-crm.ts:

---

2. Authentication & Configuration

The forwarder authenticates against the external CRM using HTTP Basic Authentication:

• Endpoint URL (HTI_CRM_URL): In development/staging, defaults to https://crm.staging.hightechinsurance.com/api/landing-page/save-leads. In production, points to the live enterprise CRM ingestion gateway.
• Username (HTI_CRM_USERNAME): API service account username.
• Password (HTI_CRM_PASSWORD): API service account password.
• Authorization Header: Constructed as Authorization: Basic ${Buffer.from(username + ":" + password).toString("base64")}.

---

3. Forwarded Payload Schema (HtiCrmPayload)

The payload structure delivered to the CRM is defined as follows:

export type HtiCrmPayload = {
  firstName: string;
  lastName?: string;
  email: string;
  phoneNumber: string;
  state: string;
  companyName?: string;
  usDot?: string | null;
  source?: string;
  leadStage?: string;
};

3.1 Sample JSON Payload

{
  "firstName": "Alejandro",
  "lastName": "Morales",
  "email": "alejandro@moraleshaul.com",
  "phoneNumber": "+12105553344",
  "state": "TX",
  "companyName": "Morales Heavy Haulage LLC",
  "usDot": "3910291",
  "source": "Web Application Quote Form",
  "leadStage": "New Inbound Quote"
}

---

4. Failure Capture Mechanics (CrmFailedLead)

If the CRM returns an HTTP error code (e.g. 500 Internal Server Error, 502 Bad Gateway, 401 Unauthorized), if the network socket hangs up, or if the CRM response JSON includes { success: false }, the forwarder intercepts the failure and creates an entry in CrmFailedLead:

// Failure Capture in src/lib/hti-crm.ts
try {
  const response = await fetch(url, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Basic ${basicAuth}`,
    },
    body: JSON.stringify(payload),
  });

  httpStatus = response.status;

  if (!response.ok) {
    const body = await response.text();
    errorMessage = `HTTP ${response.status}: ${body}`;
  } else {
    const data = await response.json();
    if (data?.success === false) {
      errorMessage = data?.message || "CRM returned success:false";
    }
  }
} catch (err: any) {
  errorMessage = err?.message || "Network/Fetch error";
}

if (errorMessage) {
  await connectDB();
  await CrmFailedLead.create({
    mongoLeadId,
    insuranceType,
    payload,
    errorMessage,
    httpStatus,
    resolved: false,
    retryCount: 0,
  });
}

---

5. Administrative Retry & Resolution Workflow

1. Dashboard Inspection: Administrators accessing the Admin Control Center (/admin) under the HTI Leads or Overview tab see an amber warning banner if unresolved CrmFailedLead records exist.
2. Payload Inspection: Clicking on a failed record displays the exact JSON payload, original submission timestamp, target CRM endpoint, and raw HTTP error message.
3. One-Click Retry Execution: Clicking "Retry Submission" executes retryCrmLeadAction in src/server/actions/admin.actions.ts.
4. Resolution Transition: If the retry returns an HTTP 200 success response, the document is updated to { resolved: true, lastRetryAt: new Date(), $inc: { retryCount: 1 } }, removing it from the active failure queue while preserving an audit record.