One fabric. Four stages.
Integration is not a project you finish — it is infrastructure someone has to operate. APIfl0w is that someone. Below is the whole fabric: how data gets in, how it is made canonical, how it is merged, and how it comes out.
The four stages
CONNECT — operated connectors
A connector is a running service, not a script. Each one owns a scoped credential in our vault, a polling schedule or a webhook listener, an upstream rate-limit budget, and a health check that pages a human when it trips.
OAuth refresh, key rotation and re-consent flows are ours to chase. When an upstream expires a token at 03:00, the client hears about it from us — with the record gap already quantified — not from a stale dashboard.
Every new connector backfills history to the earliest date the upstream will serve, at a throttled rate that respects production limits. Backfill records are marked backfill:true so trend charts do not lie about the first week.
Retry & backoff policy
| Upstream response | Fabric action | Backoff | Ceiling |
|---|---|---|---|
429 | Honour Retry-After, requeue the window | As instructed | 6 attempts |
5xx | Retry the same window | 2ⁿ s + jitter, 2–120 s | 8 attempts |
401 / 403 | Halt connector, page on-call, notify client | None — humans only | — |
404 on a known record | Tombstone the canonical record, keep lineage | None | — |
| Timeout > 30 s | Retry with halved page size | 2ⁿ s + jitter | 5 attempts |
| Malformed payload | Dead-letter with the raw body retained 30 days | None | — |
Exhausted retries emit sync.failed and open an incident — nothing fails silently, nothing retries forever.
NORMALIZE — one canonical schema
The canonical schema is the product's spine. It is small on purpose: seven record types (lead, session, deal, order, message, ticket, spend) with a shared envelope. Adding a connector never adds a record type — it adds mappings.
| Upstream field | Canonical field | Type | Transform |
|---|---|---|---|
phone_number · contactPhone · dim_phone | phone | string | E.164; default region from connector config |
call_duration_s · talk_time | duration_s | integer | Seconds, floored |
sessionDuration | engaged_s | integer | Seconds; distinct from call duration by design |
dealValue · amount_total | value.amount | integer | Minor units; value.currency as ISO 4217 |
created · createdAt · event_time | occurred_at | string | RFC 3339, normalized to UTC |
owner_email · rep | owner | string | Lower-cased, trimmed |
| anything unmapped | — | — | Dropped and listed in the sync's dropped_fields |
Conflict-resolution rules
- Freshness wins. On a scalar collision the value with the later upstream
occurred_atis kept. - Ties break on connector precedence, an ordered list set per client at onboarding and visible in the portal.
- Losing values are never destroyed. They stay on the record's
lineageblock with their connector and sync ids. - Arrays union and de-duplicate.
sourcesonly ever grows; no system is forgotten because it reported late. - Types never coerce. A string where an integer belongs dead-letters the record and raises a mapping defect. Guessing is worse than stopping.
UNIFY — the hub
Canonical records enter the hub and are matched against what is already there. Identity is deterministic and written down — no fuzzy machine-learning entity resolution making unauditable decisions about your customers.
Per record type: lead merges on normalized phone + occurred_at window; deal on upstream id + connector; order on order number + channel.
Default 72 hours, configurable per client. Late-arriving upstream data inside the window merges; outside it, a second record is created and linked, never silently folded.
Every field carries the connector id and sync id that produced it. GET /v1/records/{id}?include=lineage returns the full provenance tree.
DELIVER — three doors, one truth
The same canonical records leave the hub three ways, and all three read from the same store — there is no reporting copy that drifts.
The operator surface: fabric health, usage by unit family, record explorer with lineage, and per-connector sync history. This is what non-engineers open.
Versioned, cursor-paginated, bearer-authenticated. Everything in the dashboard is reachable through it, because the dashboard is built on it.
Signed, retried, replayable. For anything that must react within seconds rather than wait for a poll.
Four systems in. One object out.
The response below is a real shape from the unified API: a canonical lead whose sources array proves four upstream systems were merged into a single record. Each source keeps its stream colour from the hero — the same colour story, all the way down.
curl "https://api.apifl0w.com/v1/records?type=lead&since=2026-08-01" \
-H "Authorization: Bearer ak_live_7Qd2mVx9RtL0" \
-H "APIfl0w-Version: 2026-05-01"const url = "https://api.apifl0w.com/v1/records?type=lead&since=2026-08-01";
const res = await fetch(url, {
headers: {
Authorization: `Bearer ${process.env.APIFLOW_KEY}`,
"APIfl0w-Version": "2026-05-01"
}
});
if (!res.ok) throw new Error(`apiflow ${res.status}`);
const { data, next_cursor } = await res.json();
for (const record of data) {
console.log(record.id, record.phone, record.sources);
}import os, requests
BASE = "https://api.apifl0w.com/v1"
s = requests.Session()
s.headers.update({
"Authorization": "Bearer " + os.environ["APIFLOW_KEY"],
"APIfl0w-Version": "2026-05-01",
})
cursor = None
while True:
params = {"type": "lead", "since": "2026-08-01", "limit": 200}
if cursor:
params["cursor"] = cursor
page = s.get(BASE + "/records", params=params, timeout=30).json()
for record in page["data"]:
print(record["id"], record["phone"], record["sources"])
cursor = page.get("next_cursor")
if not cursor:
breakFor everything that cannot wait for a poll.
Every event is signed with HMAC-SHA256 over the raw body using your endpoint secret, timestamped, and delivered at least once. Design your handler to be idempotent on event.id; we will occasionally deliver twice, and we would rather tell you that than pretend otherwise.
| Event | Fires when | Payload root | Retry semantics |
|---|---|---|---|
record.created | A canonical record is first written to the hub | record | 8 attempts, exponential to 6 h |
record.merged | An existing record absorbs new upstream data | record, merged_from[] | 8 attempts, exponential to 6 h |
sync.completed | A connector sync run finishes without error | sync | 4 attempts, exponential to 30 m |
sync.failed | A run exhausts its retry ceiling | sync, problem | 8 attempts, exponential to 6 h |
connector.degraded | A health check trips two consecutive windows | connector | 8 attempts, exponential to 6 h |
connector.restored | Health returns to green | connector | 4 attempts, exponential to 30 m |
schema.migrated | A canonical schema version is applied to your hub | migration | 4 attempts, exponential to 30 m |
{
"id": "evt_01J9KA4T8M2WQ6R0YB1PZ7XKD",
"type": "sync.failed",
"created_at": "2026-08-11T14:07:52Z",
"hub_version": "2026-05-01",
"data": {
"sync": {
"id": "syn_88f31d",
"connector_id": "con_seo_01",
"category": "seo",
"window_start": "2026-08-11T13:00:00Z",
"window_end": "2026-08-11T14:00:00Z",
"attempts": 8,
"records_written": 0
},
"problem": {
"type": "https://apifl0w.com/errors/upstream-unavailable",
"title": "Upstream unavailable",
"status": 503,
"detail": "Upstream returned 503 on all 8 attempts; window requeued for the next cycle."
}
}
}
import crypto from "node:crypto";
// Verify before you parse. Use the RAW body, not the parsed object.
export function verify(rawBody, header, secret) {
const [tsPart, sigPart] = header.split(",");
const timestamp = tsPart.split("=")[1];
const signature = sigPart.split("=")[1];
// Reject anything older than five minutes.
if (Math.abs(Date.now() / 1000 - Number(timestamp)) > 300) return false;
const expected = crypto
.createHmac("sha256", secret)
.update(timestamp + "." + rawBody)
.digest("hex");
return crypto.timingSafeEqual(
Buffer.from(expected),
Buffer.from(signature)
);
}
The part nobody writes on a slide.
Excerpts from the runbooks our on-call engineers actually open. Published because a client evaluating an operated service deserves to know what "operated" means.
- 03:00:41Alert fired. Upstream expired the token mid-window; the connector halted rather than guessing.
- 03:09On-call acknowledged — inside the 15-minute commitment, six minutes into the shaded window.
- 03:37Fix deployed. Re-consent completed, connector resumed at its normal cadence.
- 04:10Missed windows replayed. Record gap quantified: 1,412 canonical records, all recovered.
- 08:00Client note sent — plain language, with the gap figure and the cause, before anyone asked.
- 01 AugService credit applied to the next invoice with its incident number attached. See the credit line →
2026-05-01) and pinned per client. New versions run side by side for 90 days; both are readable through the API during that window.
Your credentials are the most sensitive thing we hold.
TLS 1.2+ everywhere
Every hop — upstream fetches, hub writes, API reads, webhook deliveries — runs over TLS 1.2 or better with modern cipher suites.
AES-256, keys rotated quarterly
Canonical records, raw payload archives and dead letters are encrypted at rest with AES-256.
Upstream secrets are write-only to us
Connector runtimes fetch secrets by reference at execution time; no engineer can read a stored secret.
We ask for the smallest scope that works
Read-only scopes wherever the upstream offers them, per-connector service accounts, and no shared admin logins.
Region pinned at onboarding
Your hub is pinned to a single region — US or EU — chosen at onboarding, and nothing leaves it.
Export is a feature, not a favour
Full canonical export in JSONL, with lineage, on request and at termination — no fee, no retention hostage-taking.
Walk us through your stack.
Bring the systems, the credentials question, and the report nobody wants to build again. We will map it to the canonical schema on the call.