You might consider altering your voicemail recording to say that if you don’t have the number saved, a voicemail is required for a call back.

Mine is, “Due to a dramatic increase in scam callers, I don’t answer calls from numbers I don’t recognize. If this is a legitimate call, leave a message or send me a text and I’ll get back to you.”