Syncing Data

If you maintain a local database of CivicMarketplace data, use incremental syncing to stay up to date without re-fetching everything.

How incremental sync works

Initial sync: Page through all records using cursor pagination
Subsequent syncs: Use updated_since to fetch only records that changed since your last sync

Step 1: Initial full sync

Fetch all contracts, paginating until next_cursor is null:

async function initialSync(apiKey) {
	const contracts = [];
	let cursor = null;

	do {
		const url = new URL("https://prod-api.civicmarketplace.com/v1/contracts");
		url.searchParams.set("limit", "100");
		if (cursor) url.searchParams.set("cursor", cursor);

		const res = await fetch(url, {
			headers: { Authorization: `Bearer ${apiKey}` },
		});
		const page = await res.json();

		contracts.push(...page.data);
		cursor = page.meta.next_cursor;
	} while (cursor);

	// Store contracts and record the current timestamp
	await saveContracts(contracts);
	await saveLastSyncTime(new Date().toISOString());
}

Step 2: Incremental updates

On subsequent syncs, pass your last sync timestamp as updated_since:

async function incrementalSync(apiKey) {
	const lastSync = await getLastSyncTime();
	const updated = [];
	let cursor = null;

	do {
		const url = new URL("https://prod-api.civicmarketplace.com/v1/contracts");
		url.searchParams.set("limit", "100");
		url.searchParams.set("updated_since", lastSync);
		if (cursor) url.searchParams.set("cursor", cursor);

		const res = await fetch(url, {
			headers: { Authorization: `Bearer ${apiKey}` },
		});
		const page = await res.json();

		updated.push(...page.data);
		cursor = page.meta.next_cursor;
	} while (cursor);

	await upsertContracts(updated);
	await saveLastSyncTime(new Date().toISOString());
}

Sync schedule recommendations

Use case	Recommended interval
Real-time dashboard	Every 5 minutes
Daily reporting	Once per day
Search index	Every 15-30 minutes

The updated_since filter works with cursor pagination, so even if thousands of records changed, you can page through them safely.

Syncing suppliers

The same pattern works for suppliers:

curl "https://prod-api.civicmarketplace.com/v1/suppliers?updated_since=2026-04-20T00:00:00Z&limit=100" \
  -H "Authorization: Bearer cmp_live_YOUR_KEY"

Get Started

AI Connector (MCP)

Core Concepts

Guides

About

Overview

Contracts

Solicitations

Suppliers

Taxonomy

How incremental sync works

Step 1: Initial full sync

Step 2: Incremental updates

Sync schedule recommendations

Syncing suppliers

​How incremental sync works

​Step 1: Initial full sync

​Step 2: Incremental updates

​Sync schedule recommendations

​Syncing suppliers

How incremental sync works

Step 1: Initial full sync

Step 2: Incremental updates

Sync schedule recommendations

Syncing suppliers