One fabric. Four stages.
Integration is not a project you finish — it is infrastructure someone has to operate. APIfl0w is that someone. Below is the whole fabric: how data gets in, how it is made canonical, how it is merged, and how it comes out.
The four stages
CONNECT — operated connectors
A connector is a running service, not a script. Each one owns a scoped credential in our vault, a polling schedule or a webhook listener, an upstream rate-limit budget, and a health check that pages a human when it trips.
OAuth refresh, key rotation and re-consent flows are ours to chase. When an upstream expires a token at 03:00, the client hears about it from us — with the record gap already quantified — not from a stale dashboard.
Every new connector backfills history to the earliest date the upstream will serve, at a throttled rate that respects production limits. Backfill records are marked backfill:true so trend charts do not lie about the first week.
Retry & backoff policy
| Upstream response | Fabric action | Backoff | Ceiling |
|---|---|---|---|
429 | Honour Retry-After, requeue the window | As instructed | 6 attempts |
5xx | Retry the same window | 2ⁿ s + jitter, 2–120 s | 8 attempts |
401 / 403 | Halt connector, page on-call, notify client | None — humans only | — |
404 on a known record | Tombstone the canonical record, keep lineage | None | — |
| Timeout > 30 s | Retry with halved page size | 2ⁿ s + jitter | 5 attempts |
| Malformed payload | Dead-letter with the raw body retained 30 days | None | — |
Exhausted retries emit sync.failed and open an incident. Nothing fails silently, and nothing retries forever.
NORMALIZE — one canonical schema
The canonical schema is the product's spine. It is small on purpose: seven record types (lead, session, deal, order, message, ticket, spend) with a shared envelope. Adding a connector never adds a record type — it adds mappings.
| Upstream field | Canonical field | Type | Transform |
|---|---|---|---|
phone_number · contactPhone · dim_phone | phone | string | E.164; default region from connector config |
call_duration_s · talk_time | duration_s | integer | Seconds, floored |
sessionDuration | engaged_s | integer | Seconds; distinct from call duration by design |
dealValue · amount_total | value.amount | integer | Minor units; value.currency as ISO 4217 |
created · createdAt · event_time | occurred_at | string | RFC 3339, normalized to UTC |
owner_email · rep | owner | string | Lower-cased, trimmed |
| anything unmapped | — | — | Dropped and listed in the sync's dropped_fields |
Conflict-resolution rules
- Freshness wins. On a scalar collision the value with the later upstream
occurred_atis kept. - Ties break on connector precedence, an ordered list set per client at onboarding and visible in the portal.
- Losing values are never destroyed. They stay on the record's
lineageblock with their connector and sync ids. - Arrays union and de-duplicate.
sourcesonly ever grows; no system is forgotten because it reported late. - Types never coerce. A string where an integer belongs dead-letters the record and raises a mapping defect. Guessing is worse than stopping.
UNIFY — the hub
Canonical records enter the hub and are matched against what is already there. Identity is deterministic and written down — no fuzzy machine-learning entity resolution making unauditable decisions about your customers.
Per record type: lead merges on normalized phone + occurred_at window; deal on upstream id + connector; order on order number + channel.
Default 72 hours, configurable per client. Late-arriving upstream data inside the window merges; outside it, a second record is created and linked, never silently folded.
Every field carries the connector id and sync id that produced it. GET /v1/records/{id}?include=lineage returns the full provenance tree.
DELIVER — three doors, one truth
The same canonical records leave the hub three ways, and all three read from the same store — there is no reporting copy that drifts.
The operator surface: fabric health, usage by unit family, record explorer with lineage, and per-connector sync history. This is what non-engineers open.
Versioned, cursor-paginated, bearer-authenticated. Everything in the dashboard is reachable through it, because the dashboard is built on it.
Signed, retried, replayable. For anything that must react within seconds rather than wait for a poll.
Four systems in. One object out.
The response below is a real shape from the unified API: a canonical lead whose sources array proves four upstream systems were merged into a single record. Each source keeps its stream colour from the hero — the same colour story, all the way down.
curl "https://api.apifl0w.com/v1/records?type=lead&since=2026-08-01" \
-H "Authorization: Bearer ak_live_7Qd2mVx9RtL0" \
-H "APIfl0w-Version: 2026-05-01"const url = "https://api.apifl0w.com/v1/records?type=lead&since=2026-08-01";
const res = await fetch(url, {
headers: {
Authorization: `Bearer ${process.env.APIFLOW_KEY}`,
"APIfl0w-Version": "2026-05-01"
}
});
if (!res.ok) throw new Error(`apiflow ${res.status}`);
const { data, next_cursor } = await res.json();
for (const record of data) {
console.log(record.id, record.phone, record.sources);
}import os, requests
BASE = "https://api.apifl0w.com/v1"
s = requests.Session()
s.headers.update({
"Authorization": "Bearer " + os.environ["APIFLOW_KEY"],
"APIfl0w-Version": "2026-05-01",
})
cursor = None
while True:
params = {"type": "lead", "since": "2026-08-01", "limit": 200}
if cursor:
params["cursor"] = cursor
page = s.get(BASE + "/records", params=params, timeout=30).json()
for record in page["data"]:
print(record["id"], record["phone"], record["sources"])
cursor = page.get("next_cursor")
if not cursor:
breakFor everything that cannot wait for a poll.
Every event is signed with HMAC-SHA256 over the raw body using your endpoint secret, timestamped, and delivered at least once. Design your handler to be idempotent on event.id; we will occasionally deliver twice, and we would rather tell you that than pretend otherwise.
| Event | Fires when | Payload root | Retry semantics |
|---|---|---|---|
record.created | A canonical record is first written to the hub | record | 8 attempts, exponential to 6 h |
record.merged | An existing record absorbs new upstream data | record, merged_from[] | 8 attempts, exponential to 6 h |
sync.completed | A connector sync run finishes without error | sync | 4 attempts, exponential to 30 m |
sync.failed | A run exhausts its retry ceiling | sync, problem | 8 attempts, exponential to 6 h |
connector.degraded | A health check trips two consecutive windows | connector | 8 attempts, exponential to 6 h |
connector.restored | Health returns to green | connector | 4 attempts, exponential to 30 m |
schema.migrated | A canonical schema version is applied to your hub | migration | 4 attempts, exponential to 30 m |
{
"id": "evt_01J9KA4T8M2WQ6R0YB1PZ7XKD",
"type": "sync.failed",
"created_at": "2026-08-11T14:07:52Z",
"hub_version": "2026-05-01",
"data": {
"sync": {
"id": "syn_88f31d",
"connector_id": "con_seo_01",
"category": "seo",
"window_start": "2026-08-11T13:00:00Z",
"window_end": "2026-08-11T14:00:00Z",
"attempts": 8,
"records_written": 0
},
"problem": {
"type": "https://apifl0w.com/errors/upstream-unavailable",
"title": "Upstream unavailable",
"status": 503,
"detail": "Upstream returned 503 on all 8 attempts; window requeued for the next cycle."
}
}
}
import crypto from "node:crypto";
// Verify before you parse. Use the RAW body, not the parsed object.
export function verify(rawBody, header, secret) {
const [tsPart, sigPart] = header.split(",");
const timestamp = tsPart.split("=")[1];
const signature = sigPart.split("=")[1];
// Reject anything older than five minutes.
if (Math.abs(Date.now() / 1000 - Number(timestamp)) > 300) return false;
const expected = crypto
.createHmac("sha256", secret)
.update(timestamp + "." + rawBody)
.digest("hex");
return crypto.timingSafeEqual(
Buffer.from(expected),
Buffer.from(signature)
);
}
The part nobody writes on a slide.
Excerpts from the runbooks our on-call engineers actually open. Published because a client evaluating an operated service deserves to know what "operated" means.
connector.degraded, and surface an amber ring on the client's fabric strip with a plain-language note — never a stack trace.
2026-05-01) and pinned per client. New versions run side by side for 90 days; both are readable through the API during that window. No client is ever migrated on a Friday, and no migration is ever silent.
Your credentials are the most sensitive thing we hold.
TLS 1.2+ everywhere
Every hop — upstream fetches, hub writes, API reads, webhook deliveries — runs over TLS 1.2 or better with modern cipher suites. Plaintext transport is not configurable, including for internal service-to-service traffic.
AES-256, keys rotated quarterly
Canonical records, raw payload archives and dead letters are encrypted at rest with AES-256. Encryption keys live in a managed KMS with quarterly rotation and no standing human access.
Upstream secrets are write-only to us
Client-supplied upstream credentials go into a sealed vault. Connector runtimes fetch them by reference at execution time; no engineer can read a stored secret, and every fetch is logged with an actor, a purpose and a timestamp.
We ask for the smallest scope that works
Read-only scopes wherever the upstream offers them, per-connector service accounts, and no shared admin logins. If a platform only offers a full-access key, we say so in writing before you issue one.
Region pinned at onboarding
Your hub is pinned to a single region — US or EU — chosen at onboarding. Canonical records, archives and backups never leave it. Cross-region replication is opt-in and contractual, never a default.
Export is a feature, not a favour
Full canonical export in JSONL, with lineage, on request and at termination — no fee, no retention hostage-taking. Raw payload archives are included where the upstream licence permits redistribution to you.
Walk us through your stack.
Bring the systems, the credentials question, and the report nobody wants to build again. We will map it to the canonical schema on the call.