All the arguments about wearing colour, other club crests and, sadly, "daytrippers" are just a symptom of that frustration.