The UK's "most remote pub" has axed a Harry Potter-themed Halloween dinner party after staff were subjected to "inappropriate grief".