How to assess AI capability in a commercial leader
Asking whether someone is good at AI produces a list of products on one side of the table and a list of adjectives on the other. The question that does the most work is what broke.
Top of the agenda for many of my briefs now is clients asking me whether someone is good at AI. It is a question with no answer, and it produces a list of products on one side of the table and a list of adjectives on the other. The candidate who sounds best is frequently the one who has built the least. I know, but bear with me.
Nobody puts it quite like that, of course. It usually turns up as identity. We are an AI first business, so it ought to run through the commercial side too. That is almost never the real requirement, it is just the first thing there is language for.
Or it turns up as a hedge. Someone who can bring AI in the right way, not AI for its own sake. Honestly, that second half is the more useful one, because what they are asking for is judgement and they have not quite got the words for it yet.
The version I hear most often now is not on the specification at all. It comes up in passing, usually late in a conversation, when someone in the C-suite tells me that they have taken to pulling together their own data rather than going to the person whose job that is. They never say it as a requirement. It is the requirement though.
The most honest version I have heard came from a Head of Talent who made the case for it properly, and then said in the same breath that she was not sure it would survive alongside everything else on the list. She is right, and that is where the useful conversation starts.
What actually separates people
Four kinds of answers come back. Once you have heard them a few times you can tell within a minute or two which one you are getting.
The first is about them. Their day, their workload, how much better it all feels. Which is fine, and I am not dismissing it. Nothing has been built and nothing exists at the end of that story that did not exist before it.
The second is about something they bought. They picked it, made it fit, had a vendor alongside them. What matters here is never the product, it is why they bought instead of building, and the answer is usually IT governance or time. Added to that, the best piece of risk thinking I have heard on this subject came from somebody whose company had built almost nothing at all. He could tell me exactly why it would not work for them yet. That was great judgement, not a gap.
Third, and this is where it changes, is the person who talks about something that exists and that other people now rely on. The reliance is the line. Not code, not tooling. They know what it costs and what it gets wrong and what happens when it falls over, because their phone goes when it does.
Then a fourth group, who have stopped talking about the thing that was built and talk about the rules instead. Who decides what gets built. Who is allowed to write where. What happens when the bill runs away from you. Almost all of them have already replaced something they built themselves, and they can tell you the economics of why.
None of that is a ranking, and I would not hire in that order.
The question that does the most work
Ask what broke.
Everybody can describe a benefit, and they are not lying when they do. Work that took days now takes hours, the team is quicker, we have rolled it out everywhere, I could not go back. All true, probably, but listen to what is not there. Nothing has a name. No system it reads from or writes to. The saving is not attached to any particular piece of work, so it could be describing anything. Nothing went wrong, nothing cost anything, and there is nobody else in the story at all.
The answer from somebody who has actually built something sounds different straight away. Everything in it has a name. A platform, a system it pulls from, somewhere it writes to, a checking step in the middle with a person signing it off. The numbers are about cost and about what it got wrong, not only about what it saved. Somewhere in it there is a reversal, something built and then abandoned, or bought and then dropped, and the reason is always so specific to their own stack that you could not invent it. And there is a refusal in there too. At some point they decided not to do something. The best version I have heard was a candidate who would let it read from the CRM but would absolutely not let it write back.
Here is the part that surprises clients. The people who have built something can be a bit bored of the subject or at least seem so. They play their own work down while describing it, call it simple, say it is not really AI, say the interesting bit was the data. The enthusiastic ones, more often than not, have not built much. That is the opposite of what an interview rewards, and it costs good people offers.
If you are the one being asked, it works the same way round. Playing it down is read correctly by anyone who knows the subject, so there is nothing to be gained by talking it up. The habit that does the damage is the other one, describing work that was yours as something a team did, until there is nothing left in the story with your name on it.
There is a reason to expect something to have gone wrong in any honest account. Gartner predicted back in June 2025 that more than forty per cent of agentic AI projects would be cancelled before the end of 2027, and they have not moved off it since. What they blamed was cost, unclear value and not enough control over the risk. None of that is the technology failing. Their 2026 work puts seventeen per cent of organisations as having actually deployed agents, against more than sixty per cent who expect to within two years, which tells you most of the market is talking about something it has not done yet. So if you are hiring, anyone genuinely close to this has something that went badly to tell you about. Not having one is not modesty.
Zanda, a firm doing this in finance, published something of the same shape this month, written by Andrew Waters, about when the AI-native hire is the wrong hire. Different function, same quarter, same problem underneath. Two firms landing on it separately says the pattern is real rather than theoretical.
What to hire for
What is worth buying is set by what your business can take, not by what is out there. Bring in someone two stages ahead of the organisation around them and they will spend their first year explaining instead of building. Most of them leave before the explaining is done. Asking for the most advanced person on the market is usually asking for the wrong one.
Which leaves a question most companies have never been asked, and it is worth settling before any of this goes anywhere near a brief. Working that out is part of what Animate’s Pre-Search Diagnostic produces: a specification tested against the people who actually exist, agreed by everyone who is going to interview, before the first approach.
If you are about to write this into a brief, send me what you have and I will tell you where a shortlist would actually sit. darren@animatesearch.com
