Customs Data Pipeline
A prospect data pipeline for a customs compliance company. It collects public lists of customs brokers, cleans and checks them, then feeds the sales CRM and outreach.
Every record checked before sales sees it.
- Client
- Customs compliance company (anonymized)
- Status
- In build, tested against a stand-in CRM
- Role
- Systems design and build
Broker records, checked firstMock-up
- Broker from the US registryMatchEmailConsentPassed
- Two names, one websiteMatchEmailConsentDuplicate domain
- Contact on a temp inboxMatchEmailConsentThrowaway email
- Broker in CanadaMatchEmailConsentNo consent: phone only
- Same broker, two listsMatchEmailConsentMerged
- Email with no mail serverMatchEmailConsentStopped
Mock-up. The real platform is under NDA, so these screens use made-up data.
How every record gets checked
Bad data in a CRM means wasted calls and emails that break the law. So nothing gets in until it passes.
Raw rows kept
Every row is stored untouched, with where and when it was captured.
Duplicates caught
Records merge on name plus country. Companies that share a website get flagged for a person to check.
Emails tested
Each email is checked for format, a working mail server and throwaway addresses.
Anti-spam law built in
Canadian contacts are phone only unless consent is recorded. An email found on a company's own site is stored with the page and date as proof.
A polite site reader
The tool that reads company websites follows each site's robots.txt rules.
Paid checks need a yes
Paid email verification only runs on a batch that has been approved.
The problem
The sales team needed a list of customs brokers to call and email. The public lists are big and messy, with duplicates, dead emails and contacts that Canadian law says you cannot email without consent.
What we built
A pipeline that loads the public US and Canadian broker registries and a master sheet, reads company websites, checks every record and exports clean call lists, visit lists and CRM records.
Features
- Loaders for the public US and Canadian broker registries
- A loader for the master prospect sheet
- A website reader for contact details
- Call and visit lists
- CRM export
- n8n workflows for outreach
- A CRM API spec
- Tests for every step
Stack
- Python 3.12
- httpx
- dnspython
- phonenumbers
- SQLite
- n8n
The goal: a CRM the sales team can trust, with the law handled before anyone hits send.
Our role
We designed the pipeline and built the loaders, the checks, the website reader, the workflows and the CRM spec. It is in build and tested against a stand-in CRM.
Client anonymized. More technical detail available on request.
About the example
The screen at the top of this page is a mock-up with made-up data. The real one holds client information, so it stays private.
Need your data checked?
Systems design, from approx. $250/hour, scoped per engagement.
