---

A number on a dashboard carries authority it has not always earned. When a CRM shows a deal at “82% likely to close,” managers adjust forecasts, reps change their priorities and nobody pauses to ask where 82 came from. The following questions are what a careful analyst would ask before letting a score drive decisions. You do not need a statistics degree to ask them.

1. What is it actually predicting?

“Lead score” can mean likelihood to reply, likelihood to book a meeting, likelihood to become a customer, or likelihood to become a large customer. These are different outcomes. Find the exact definition and time window. A model trained to predict who replies to emails may rank cheap, quick-closing leads highest, which is not what you want if you sell long, expensive contracts.

2. What data does it use?

Ask which fields and activities feed the score. Three things to watch for. First, fields that only get filled in late in the process, such as a proposal amount, can leak the answer into the score and make it look more accurate than it is. Second, fields that are sparsely or unevenly filled bias the score toward the reps who log diligently. Third, proxies for sensitive characteristics can lead to unfair results, depending on the context and the regulations that apply to you.

3. How was it checked?

A vendor should be able to explain how the model was evaluated: on which data, over what period, and against what baseline. The key idea is that a score must be tested on outcomes the model has not seen during training. A model that looks excellent on the data it learned from can fall apart on next quarter’s leads.

Ask also what a simple alternative would have achieved. If sorting leads by recent activity gets close to the same ranking, the model is not adding much. Do not accept a single accuracy figure as proof; ask what happens at the top of the ranked list, since that is where your reps spend their time.

4. Is it calibrated for your business?

A score is calibrated when “70” really means about seven in ten such records convert. Many scores are only rankings: higher is better, but the numbers are not true probabilities. Using an uncalibrated score as a probability in a forecast leads to over- or under-statement. If you intend to sum scores into a forecast, ask for evidence they behave like probabilities in your own history.

5. How fresh is it?

Scores that update nightly may lag a deal that changed yesterday. Scores that update in real time may swing wildly on a single email. Learn the refresh cadence and show it on screen. Reps should be able to see when the score was last computed.

6. Why did this record get this score?

A reason helps you act. “High score because three stakeholders engaged and the pricing page was viewed twice” tells a rep what to do. A bare number does not. Explanations from models are approximations, so treat them as clues, not proofs, but insist on them.

7. When does it make mistakes?

Every model has weak spots. Ask where this one is known to struggle: new segments, unusual deal types, small samples, accounts with little history. Cold-start cases, where there is almost no data, are frequent in sales. A score for a brand-new lead with a bare record is mostly noise, and the tool should say so or show a low-confidence marker.

8. Who can override it, and what happens then?

Reps and managers should be able to disagree with the score and record why. Overrides are valuable data. If reps override the same type of score again and again, the model is missing something. If nobody can override, the score is a rule disguised as insight.

9. What decision will it change?

Write down the decision tied to the score. Examples: who gets called first, which leads are routed to senior reps, which deals are examined in the weekly review. If a score changes no decision, it is decoration. If a wrong score would trigger something costly or irreversible, such as dropping a lead from follow-up entirely, add a human check.

A fast reference

Question	A good answer sounds like	A worrying answer sounds like
What does it predict?	A named outcome in a defined window	“Quality” or “propensity”
What data feeds it?	A list of fields you can inspect	“Proprietary signals”
How was it tested?	Held-out data, a baseline comparison	One headline accuracy number
Why this score?	Top contributing factors shown	A number only
Where does it fail?	Known weak segments listed	“It works everywhere”
Can we override?	Yes, and it is logged	Not supported

A hypothetical check

A manager notices that deals scored above a certain level close often, and a small set of high-scoring deals are stuck. She exports the last quarter’s closed deals, groups them by the score band at the time of creation, and compares outcomes. The top band does close more often, but when she looks at score band at the time of the first meeting instead of at the end, the difference shrinks sharply. Part of the score was reading late-stage activity. She asks the vendor which inputs are timestamped, and the team agrees to use the score only as a prioritization aid for early-stage leads, not as a forecast input.

Frequently Asked Questions

Should we ignore scores we cannot explain? Not entirely, but give them less weight, and avoid using them for high-stakes or irreversible decisions until you understand them.

What if the vendor will not share how the model works? You may not get the algorithm, but you should get the inputs, the evaluation method and known limitations. Treat refusal to share those as a risk.

Can we build our own check without a data team? Yes. Export historical records with scores at a known point in time and outcomes, group them into bands and compare close rates. A spreadsheet is enough for a first look.

Do scores replace rep judgment? They support it. Experienced reps often notice things the model cannot see, such as a buyer’s tone or an internal reorganization.

How often should we revalidate? At least quarterly, and whenever your pricing, segments or process change significantly.

Next Step

Pick the one score your team uses most and answer the first three questions for it in writing. If you cannot answer them, ask your vendor or administrator this week, and hold off on using that score in the forecast until you can.