This happens to me all the time in my single-player world. Someone in X's comments explained what's happening-if the village center is too far away from where the raid spawns, the pillagers' primary goal is to pathfind closer, which, in cases like this, they can't do, rendering them essentially passive. Doesn't work for evokers, witches or ravagers, though.

wth THAT IS A WIERD GLITCH EVEN FOR HERMIT CRAFT