05 / AFFILIATE PRODUCT SYNC / DATA PROCESSING

WISH2SHARE

ROLELead EngineerTYPEDirect ContractYEAR2025
AFFILIATE FEEDS
→
NORMALIZE
→
PROCESS
→
CATALOGUE
01 / CONTEXT

A multi-network affiliate product-sync engine using delta-hash deduplication and resumable processing to handle feeds larger than the platform's execution window.

Prototype sync logic worked on small feeds but failed against production-scale affiliate data. Several networks published different formats on different schedules, while Vercel imposed a hard 300-second execution limit on each invocation.

02 / THE HARD PART
The problem wasn't parsing a large file. It was designing the work so an arbitrarily large feed could finish reliably without requiring one serverless request to survive until the entire job completed.
03 / WHAT SHIPPED
Delta-hash product deduplication
Time-boxed processing
Cursor-based resume across invocations
Per-network feed adapters
Unified affiliate product model
Admin sync controls and status visibility
04 / STACK
Next.jsTypeScriptNode.jsFirestoreVercel

NEXT / 06

MYFILETRACKER