If you maintain a local database of CivicMarketplace data, use incremental syncing to stay up to date without re-fetching everything.
How incremental sync works
- Initial sync: Page through all records using cursor pagination
- Subsequent syncs: Use
updated_since to fetch only records that changed since your last sync
Step 1: Initial full sync
Fetch all contracts, paginating until next_cursor is null:
async function initialSync(apiKey) {
const contracts = [];
let cursor = null;
do {
const url = new URL("https://prod-api.civicmarketplace.com/v1/contracts");
url.searchParams.set("limit", "100");
if (cursor) url.searchParams.set("cursor", cursor);
const res = await fetch(url, {
headers: { Authorization: `Bearer ${apiKey}` },
});
const page = await res.json();
contracts.push(...page.data);
cursor = page.meta.next_cursor;
} while (cursor);
// Store contracts and record the current timestamp
await saveContracts(contracts);
await saveLastSyncTime(new Date().toISOString());
}
Step 2: Incremental updates
On subsequent syncs, pass your last sync timestamp as updated_since:
async function incrementalSync(apiKey) {
const lastSync = await getLastSyncTime();
const updated = [];
let cursor = null;
do {
const url = new URL("https://prod-api.civicmarketplace.com/v1/contracts");
url.searchParams.set("limit", "100");
url.searchParams.set("updated_since", lastSync);
if (cursor) url.searchParams.set("cursor", cursor);
const res = await fetch(url, {
headers: { Authorization: `Bearer ${apiKey}` },
});
const page = await res.json();
updated.push(...page.data);
cursor = page.meta.next_cursor;
} while (cursor);
await upsertContracts(updated);
await saveLastSyncTime(new Date().toISOString());
}
Sync schedule recommendations
| Use case | Recommended interval |
|---|
| Real-time dashboard | Every 5 minutes |
| Daily reporting | Once per day |
| Search index | Every 15-30 minutes |
The updated_since filter works with cursor pagination, so even if thousands of records changed, you can page through them safely.
Syncing suppliers
The same pattern works for suppliers:
curl "https://prod-api.civicmarketplace.com/v1/suppliers?updated_since=2026-04-20T00:00:00Z&limit=100" \
-H "Authorization: Bearer cmp_live_YOUR_KEY"
