Conversation and context aware fraud and abuse prevention agent

One embodiment provides a method comprising intercepting a voice communication, collecting multi-sensory inputs associated with the voice communication, and determining an overall risk assessment metric for the voice communication based on the multi-sensory inputs and learned signatures. The multi-sensory inputs are indicative of content of the voice communication and one or more contextual factors associated with a target of the voice communication. The overall risk assessment metric indicates a likelihood the voice communication is a scam.