Platform

One fabric. Four stages.

Integration is not a project you finish — it is infrastructure someone has to operate. APIfl0w is that someone. Below is the whole fabric: how data gets in, how it is made canonical, how it is merged, and how it comes out.

API version 2026-05-01 p50 sync latency 412 ms backfill included
UPSTREAM NORMALIZE CANONICAL

The four stages

STATION 01

CONNECT — operated connectors

A connector is a running service, not a script. Each one owns a scoped credential in our vault, a polling schedule or a webhook listener, an upstream rate-limit budget, and a health check that pages a human when it trips.

Auth handling

OAuth refresh, key rotation and re-consent flows are ours to chase. When an upstream expires a token at 03:00, the client hears about it from us — with the record gap already quantified — not from a stale dashboard.

Backfill

Every new connector backfills history to the earliest date the upstream will serve, at a throttled rate that respects production limits. Backfill records are marked backfill:true so trend charts do not lie about the first week.

Retry & backoff policy

How the fabric responds to upstream failures
Upstream responseFabric actionBackoffCeiling
429Honour Retry-After, requeue the windowAs instructed6 attempts
5xxRetry the same window2ⁿ s + jitter, 2–120 s8 attempts
401 / 403Halt connector, page on-call, notify clientNone — humans only—
404 on a known recordTombstone the canonical record, keep lineageNone—
Timeout > 30 sRetry with halved page size2ⁿ s + jitter5 attempts
Malformed payloadDead-letter with the raw body retained 30 daysNone—

Exhausted retries emit sync.failed and open an incident — nothing fails silently, nothing retries forever.

STATION 02

NORMALIZE — one canonical schema

The canonical schema is the product's spine. It is small on purpose: seven record types (lead, session, deal, order, message, ticket, spend) with a shared envelope. Adding a connector never adds a record type — it adds mappings.

Example field mappings from upstream systems into canonical fields
Upstream fieldCanonical fieldTypeTransform
phone_number · contactPhone · dim_phonephonestringE.164; default region from connector config
call_duration_s · talk_timeduration_sintegerSeconds, floored
sessionDurationengaged_sintegerSeconds; distinct from call duration by design
dealValue · amount_totalvalue.amountintegerMinor units; value.currency as ISO 4217
created · createdAt · event_timeoccurred_atstringRFC 3339, normalized to UTC
owner_email · repownerstringLower-cased, trimmed
anything unmapped——Dropped and listed in the sync's dropped_fields

Conflict-resolution rules

  • Freshness wins. On a scalar collision the value with the later upstream occurred_at is kept.
  • Ties break on connector precedence, an ordered list set per client at onboarding and visible in the portal.
  • Losing values are never destroyed. They stay on the record's lineage block with their connector and sync ids.
  • Arrays union and de-duplicate. sources only ever grows; no system is forgotten because it reported late.
  • Types never coerce. A string where an integer belongs dead-letters the record and raises a mapping defect. Guessing is worse than stopping.
STATION 03

UNIFY — the hub

Canonical records enter the hub and are matched against what is already there. Identity is deterministic and written down — no fuzzy machine-learning entity resolution making unauditable decisions about your customers.

Merge keys

Per record type: lead merges on normalized phone + occurred_at window; deal on upstream id + connector; order on order number + channel.

Dedup window

Default 72 hours, configurable per client. Late-arriving upstream data inside the window merges; outside it, a second record is created and linked, never silently folded.

Lineage

Every field carries the connector id and sync id that produced it. GET /v1/records/{id}?include=lineage returns the full provenance tree.

Why lineage is not optional When a branch manager and a marketing lead disagree about a number, the argument is never about the number. It is about which system produced it. Lineage ends that conversation in one click, which is the only reason anyone trusts a unified dashboard six months in.
STATION 04

DELIVER — three doors, one truth

The same canonical records leave the hub three ways, and all three read from the same store — there is no reporting copy that drifts.

Unified dashboard

The operator surface: fabric health, usage by unit family, record explorer with lineage, and per-connector sync history. This is what non-engineers open.

Unified REST API

Versioned, cursor-paginated, bearer-authenticated. Everything in the dashboard is reachable through it, because the dashboard is built on it.

Webhooks

Signed, retried, replayable. For anything that must react within seconds rather than wait for a poll.

One API

Four systems in. One object out.

The response below is a real shape from the unified API: a canonical lead whose sources array proves four upstream systems were merged into a single record. Each source keeps its stream colour from the hero — the same colour story, all the way down.

records · v1
curl "https://api.apifl0w.com/v1/records/rec_01J9K7M2QXB4S0E7VN3TD8HZ" -H "Authorization: Bearer ak_live_7Qd2mVx9RtL0" { "id": "rec_01J9K7M2QXB4S0E7VN3TD8HZ", "type": "lead", "occurred_at": "2026-08-11T14:02:17Z", "phone": "+14155550142", "duration_s": 214, "engaged_s": 96, "value": { "amount": 482000, "currency": "USD" }, "sources": ["call_tracking", "seo", "analytics", "crm"], "lineage": { "phone": "con_calltrack_01 @ syn_88f21c", "engaged_s": "con_analytics_02 @ syn_88f23e", "value": "con_crm_01 @ syn_88f2a0" }, "hub_version": "2026-05-01"}
list leads
curl "https://api.apifl0w.com/v1/records?type=lead&since=2026-08-01" \
  -H "Authorization: Bearer ak_live_7Qd2mVx9RtL0" \
  -H "APIfl0w-Version: 2026-05-01"
Webhooks

For everything that cannot wait for a poll.

Every event is signed with HMAC-SHA256 over the raw body using your endpoint secret, timestamped, and delivered at least once. Design your handler to be idempotent on event.id; we will occasionally deliver twice, and we would rather tell you that than pretend otherwise.

Webhook event catalog
EventFires whenPayload rootRetry semantics
record.createdA canonical record is first written to the hubrecord8 attempts, exponential to 6 h
record.mergedAn existing record absorbs new upstream datarecord, merged_from[]8 attempts, exponential to 6 h
sync.completedA connector sync run finishes without errorsync4 attempts, exponential to 30 m
sync.failedA run exhausts its retry ceilingsync, problem8 attempts, exponential to 6 h
connector.degradedA health check trips two consecutive windowsconnector8 attempts, exponential to 6 h
connector.restoredHealth returns to greenconnector4 attempts, exponential to 30 m
schema.migratedA canonical schema version is applied to your hubmigration4 attempts, exponential to 30 m
sync.failed — delivered payload
{
  "id": "evt_01J9KA4T8M2WQ6R0YB1PZ7XKD",
  "type": "sync.failed",
  "created_at": "2026-08-11T14:07:52Z",
  "hub_version": "2026-05-01",
  "data": {
    "sync": {
      "id": "syn_88f31d",
      "connector_id": "con_seo_01",
      "category": "seo",
      "window_start": "2026-08-11T13:00:00Z",
      "window_end": "2026-08-11T14:00:00Z",
      "attempts": 8,
      "records_written": 0
    },
    "problem": {
      "type": "https://apifl0w.com/errors/upstream-unavailable",
      "title": "Upstream unavailable",
      "status": 503,
      "detail": "Upstream returned 503 on all 8 attempts; window requeued for the next cycle."
    }
  }
}
signature verification — Node
import crypto from "node:crypto";

// Verify before you parse. Use the RAW body, not the parsed object.
export function verify(rawBody, header, secret) {
  const [tsPart, sigPart] = header.split(",");
  const timestamp = tsPart.split("=")[1];
  const signature = sigPart.split("=")[1];

  // Reject anything older than five minutes.
  if (Math.abs(Date.now() / 1000 - Number(timestamp)) > 300) return false;

  const expected = crypto
    .createHmac("sha256", secret)
    .update(timestamp + "." + rawBody)
    .digest("hex");

  return crypto.timingSafeEqual(
    Buffer.from(expected),
    Buffer.from(signature)
  );
}
Operations

The part nobody writes on a slide.

Excerpts from the runbooks our on-call engineers actually open. Published because a client evaluating an operated service deserves to know what "operated" means.

INC-2026-0731 · replay 00:00
15-MIN ACK WINDOW
00:0006:0012:0018:0024:00
  • 03:00:41Alert fired. Upstream expired the token mid-window; the connector halted rather than guessing.
  • 03:09On-call acknowledged — inside the 15-minute commitment, six minutes into the shaded window.
  • 03:37Fix deployed. Re-consent completed, connector resumed at its normal cadence.
  • 04:10Missed windows replayed. Record gap quantified: 1,412 canonical records, all recovered.
  • 08:00Client note sent — plain language, with the gap figure and the cause, before anyone asked.
  • 01 AugService credit applied to the next invoice with its incident number attached. See the credit line →
RB-030 · schema migrations Canonical schema versions are dated (2026-05-01) and pinned per client. New versions run side by side for 90 days; both are readable through the API during that window.
RB-037 · connector retirement When a client drops an upstream system, its connector is paused, not deleted. Records and lineage remain queryable indefinitely; the connector stops counting as active in the next billing period.
RB-042 · dead-letter review Dead-lettered payloads are reviewed daily, not when someone notices. A mapping defect that produces more than 20 dead letters in a window becomes a ticket the same day.
Security

Your credentials are the most sensitive thing we hold.

In transit

TLS 1.2+ everywhere

Every hop — upstream fetches, hub writes, API reads, webhook deliveries — runs over TLS 1.2 or better with modern cipher suites.

At rest

AES-256, keys rotated quarterly

Canonical records, raw payload archives and dead letters are encrypted at rest with AES-256.

Credential vaulting

Upstream secrets are write-only to us

Connector runtimes fetch secrets by reference at execution time; no engineer can read a stored secret.

Least privilege

We ask for the smallest scope that works

Read-only scopes wherever the upstream offers them, per-connector service accounts, and no shared admin logins.

Data residency

Region pinned at onboarding

Your hub is pinned to a single region — US or EU — chosen at onboarding, and nothing leaves it.

Your exit

Export is a feature, not a favour

Full canonical export in JSONL, with lineage, on request and at termination — no fee, no retention hostage-taking.

Walk us through your stack.

Bring the systems, the credentials question, and the report nobody wants to build again. We will map it to the canonical schema on the call.