Aymeric's logic works for most people, most relationships. However, Finnick is pretty sure Annie would call out for him if she were in trouble, regardless of whether she knew he was anywhere around or not. He's her caretaker, her protector. It's just how things are with the two of them.

So he's inherently reluctant to walk away from her pleas, even as they become repetitive and a little uncanny, not quite right. The two others are persistent, though, so he doesn't feel like he has much choice. He either leaves with them, or he abandons them and forces them to chase after him like he's a crazy person.

"Alright..." He mutters, looking back once more before following Aymeric and Leofard back toward the ship.