Irresponsible comments influencing voters on answers: You write an answer covering multiple aspects of the issue in reasonable detail. Minutes later, someone writes a comment, "Downvoting because you start by suggesting X, but it that would lead to issue Y."
Now, you were aware of the issue Y when you wrote the answer, and had elaborated how to deal with that a couple of sentences later, so what went wrong here? The problem is that either some people have the attention span of a goldfish and hence cannot read more than 2 sentences, or that they have an irresistible urge to criticize and hence cannot have the patience to read the complete answer before commenting.
Then other users read the comment, and downvote the answer because, (duh!) the answer must be definitely wrong because a comment says so, and anyway, who has the time to read the answer and use their own brains to figure out if the criticism is valid or not?
This is made even worse by StackExchange's strange system of "hiding" comments based on upvotes, so even if you post your own comment explaining why that criticism is unjustified, it would most likely be hidden, and again, who wants to bother reading all the comments, when it is just easier to downvote by reading the top voted comment?
Unfortunately, the comment flagging mechanism isn't much helpful here either. To begin with, while the criticism may be invalid, it isn't actually rude, offensive, "chatty", etc. so flagging it will most likely not lead to its deletion. Moreover, comment deletion can be done only by a moderator, which means usually the damage has already been done by the time they get to it.
So, what can we do about it?
On the whole, nothing much. At a personal level though, we could resist the urge to comment the moment we see something we don't agree with, and actually have the patience to read and understand the complete answer.
We could also avoid passing judgements (especially, the use of "downvoting/-1 because ...") in the comments, and instead phrase our criticism as a question: "How does your solution X deal with issue Y?" Criticism phrased this way is much more constructive. Moreover, this also allows the commenter to save face, as against, "ha ha, your solution missed issue Y, look I am so clever."
Also, if we do not have the patience to read the complete answer, we have the option to ... not comment at all (shocking fact, I know!).
As an aside, this nuisance was one of the main factors that led me to retire in restricted mode from this site.