How to Test a B2B Data Vendor Before You Sign Anything

Here is how most enrichment evaluations go.

You ask three vendors for a sample. Each one sends back a file. You open them in a spreadsheet, scan a few rows, and notice that one has more filled cells than the others. That vendor wins. Six months later, your reps are complaining that half the titles are wrong and nobody can explain how the evaluation missed it.

The evaluation did not miss it. The evaluation measured fill rate, and fill rate is not accuracy. Those are different numbers and they often move in opposite directions.

This post is a method for testing data vendors properly. It works for any vendor, including us. Most of it is unglamorous, and the whole thing takes about a week of one person's time, which is a rounding error against the cost of getting it wrong on a multi-year contract.

Rule one: never evaluate on the vendor's sample

Every vendor sample is drawn from where that vendor is strongest. Nobody is being dishonest about this. If someone asks you to show what your data looks like, you show it working.

The problem is that "working" is measured on their distribution, not yours. A vendor with excellent coverage of US software companies and thin coverage of German industrials will send you a sample of US software companies. If you sell to German industrials, you have learned nothing, and you have learned it in a way that feels informative.

Your test set has to come from your data. Not a representative slice of the market. Yours, with your geographic skew, your industry concentration, your company size distribution, and your particular mix of stale and fresh records.

Five hundred records is enough. Two thousand is better if you can assemble it. Below about two hundred, random variation swamps the differences you are trying to measure.

Rule two: build a golden set, and accept that it hurts

This is the step everyone skips, and skipping it is why most evaluations are theatre.

To know whether a vendor is right, you need to know what right is. That means taking a subset of your test records, maybe a hundred, and manually establishing the truth for each one. Actual current title. Actual current employer. Actual company headcount. Verified by a human, looking at primary sources, one record at a time.

It is tedious and it takes a person a day or two. It is also the only thing that converts your evaluation from an opinion into a measurement.

Without a golden set you can compare vendors to each other, which tells you who disagrees with whom, but not who is right. Two vendors agreeing on a wrong answer looks exactly like two vendors agreeing on a right one.

A few things that make the golden set less painful:

Timestamp it. Record the date you verified each fact. Data ages, and in six months you will want to know whether a discrepancy means the vendor is wrong or your golden set is stale.

Include known-hard cases deliberately. Recent job changers. People at companies that were acquired. Common names. Non-Latin-script names. Small companies. Companies with several legal entities. If your golden set is all VPs at well-known US tech companies, every vendor will look excellent and you will have learned nothing about the hard 30% of your database.

Include records you expect to fail. People who left the workforce, companies that shut down, addresses you know are dead. A vendor that confidently returns data for a company that no longer exists has told you something important about how it handles uncertainty.

Rule three: the four numbers that matter

Fill rate is the number vendors lead with and the least useful one on this list.

Match rate. Of the records you submitted, how many came back with anything at all? This is the headline number and the one most likely to be optimized for. A vendor can raise match rate by loosening its matching threshold, which raises match rate and lowers accuracy at the same time. Never read it alone.

Accuracy. Of the records that matched, how many are correct against your golden set? This is the number that determines whether your reps trust the CRM. Measure it per field, because a vendor can be excellent on company and mediocre on title, and those failures cost you differently.

Precision on the hard cases. Accuracy measured only on the subset you deliberately made difficult. This is the number that best predicts real-world performance, because the easy records were never the problem.

Effective cost per usable record. Total spend divided by records that were both matched and correct. Not cost per call. Not cost per match. This is the only cost figure that lets you compare vendors with different pricing models, and it is usually two to three times the headline rate.

That last one deserves an example. Two vendors, both priced at the same rate per lookup. Vendor A matches 80% with 70% accuracy. Vendor B matches 55% with 95% accuracy. Vendor A looks better on the demo. But A gives you 56 usable records per hundred and B gives you 52, which is close to a tie, and A also hands you 24 confidently wrong records that your team will act on before anyone notices. The demo winner is the worse product.

One practical obstacle here: most vendors make effective cost genuinely hard to compute, because spend arrives as a monthly invoice and you are left reverse-engineering which calls produced which charges. Ask during the trial whether cost is reported per call. Renidly returns the credits consumed on every individual response, so this figure is something you read off a log rather than reconstruct from a bill three weeks later. Whoever you test, insist on being able to compute it, because a vendor that cannot show you per-call cost is a vendor whose pricing you will not understand until you are already committed.

Rule four: test the failure behavior

This is the part almost nobody tests, and it separates vendors more sharply than accuracy does.

When a vendor cannot find something, what does it do?

The good answer is that it tells you, clearly, with a signal you can branch on. A confidence score, a not-found flag, a status code that means "no result" rather than "here is a result." The bad answer is that it returns its best guess with the same shape and the same apparent authority as a real match, leaving you no way to tell them apart.

The second behavior is much worse than a lower match rate, and it does not show up in any of the four numbers above unless you go looking.

To test it, submit records you know should fail. Fake companies. People who do not exist. Malformed inputs. Then look at what comes back and ask whether your code could tell the difference between that and a genuine result. If it could not, every wrong answer that vendor produces will land in your CRM wearing the same clothes as a right one.

While you are there, check whether confidence scores are actually calibrated. Take everything the vendor marked high confidence and measure accuracy on just that subset. If high-confidence records are not meaningfully more accurate than low-confidence ones, the score is decoration and you cannot build routing logic on it.

This is the test we would most encourage you to run on us. Renidly returns a four-tier confidence value rather than a single boolean, and an explicit not-found response rather than a silent best guess, specifically so you can write different code paths for "certain," "probably," and "no idea." A vendor that gives you one undifferentiated answer forces you to treat every result at the same trust level, which in practice means treating all of them at the level of the worst one.

Rule five: freshness is a separate test, and it is the hard one

Accuracy tests tell you whether a record is right. They do not tell you how long it stays right, and in professional data that is most of the value.

People change jobs constantly. A dataset that was 95% accurate when it was assembled decays continuously, and the decay is invisible because a stale record looks exactly like a fresh one.

There are two honest ways to test this and both take time.

The retrospective method. Find twenty to thirty people in your own network or CRM who you know changed jobs in the last ninety days, because you saw it happen. Submit them. Count how many vendors have the new role. This is a small sample and it is biased toward your network, but it is direct evidence and you can run it in an afternoon.

The longitudinal method. Pull a set of records now. Pull the same records again in sixty days. Measure how many changed, and separately, verify a sample of the changes against reality. A vendor whose records never change is not stable, it is stale. A vendor whose records change constantly may be updating or may be churning noise, and the only way to tell is to check whether the changes were real.

This is the test that most reliably separates vendors, and it is the one most evaluations skip because it does not fit inside a two-week procurement window. Start it on day one of your evaluation and let it run while you do everything else.

Worth asking each vendor a specific question here: is freshness something you choose per request, or something baked into the whole dataset? Renidly separates the two, so a bulk read and a freshest-available resolution are different calls against the same records, which means you can test both paths on the same golden set and see exactly what the difference is worth on your data. If a vendor only offers one mode, you are accepting their judgment about a tradeoff that should be yours to make per workload.

Rule six: test the things procurement will ask about later

If you are buying for an enterprise, the evaluation is not just about data quality. Half the deals that fail at stage four fail on things nobody tested at stage two.

Ask, early, in writing:

• What happens to data you have already pulled if the contract ends?
• What happens if a person exercises a deletion right, and how does that propagate to records you have already stored?
• What is the actual uptime commitment, and what is the remedy when it is missed?
• Where can you point your legal team for the sourcing and compliance posture?
• Is there a DPA you can sign, and has anyone outside the vendor audited any of this?

A vendor that answers these quickly and specifically has been asked before. A vendor that gets vague has not, and you will be the customer who discovers what the answers are.

This is worth doing at the start rather than the end, because these questions kill deals late, after you have spent three months on an evaluation and built an integration against an API you now cannot use.

What good looks like

Rough benchmarks for B2B professional data, from working with enough of these evaluations to have opinions:

Two caveats on that table. These vary enormously by geography and industry, so a vendor that looks weak on a German industrial dataset may be strong on US software. And a vendor scoring 95% match with 70% accuracy should worry you more than one scoring 55% with 95% accuracy, for the reason in rule three.

The argument against everything above

Here is where I will argue against my own method.

This test takes about a week, and for a lot of teams that is not a week well spent. If you are buying a few thousand credits to see whether enrichment helps at all, run fifty records, eyeball the results, and move on. The rigorous version is for six-figure commitments and multi-year contracts, not for a first experiment.

And there is a real risk of over-indexing on measurement. A vendor that scores 3% better on accuracy but has no batch endpoint, unclear failure modes, and a support queue that answers in four days is the worse choice, and none of that shows up in the table above. Integration quality, error semantics, and how a vendor behaves when something breaks are worth more over a two-year contract than three points of accuracy.

The numbers exist to stop you being fooled by a demo. They are not a substitute for judgment about whether this is a company you want to depend on.

The one-line version

Test on your own data, build a golden set even though it hurts, measure accuracy rather than fill rate, deliberately submit records that should fail, and start the freshness test on day one because it is the only one that needs calendar time.

Any vendor worth signing will be comfortable with all of that. If a vendor pushes back on you running a structured test, or wants to supply the test set themselves, that reluctance is itself a result.

We would rather be tested this way than pitched. There are 100 free credits, no card required, which covers a 500-record evaluation comfortably, and if your numbers come out differently from what you expected in either direction we would genuinely rather hear about it. If you have not defined which records are worth enriching yet, the guide to precise ICP filtering covers narrowing a list before you spend anything on it, and batch enrichment at scale has the mechanics of running a large test set without hitting a rate limit halfway through.