When it comes to comments, whether or not I delete or dismiss the flags depends on several factors.
In What "comments" are not . . ., Robert Cartaino clarifies that the purpose of comments are to either improve a post or to seek clarification from an asker. Comments that attempt to answer the question or that don't further one of the aforementioned goals are subject to removal.
I generally only remove comment threads when they're auto-flagged by the Community bot or when it's clear that the comments may further degrade a borderline not-constructive question into extended discussion. Otherwise, if there are only a few comments, I may leave them.
As for comments flagged as rude/offensive, I may edit them, delete them, or dismiss them depending on my interpretation of what is rude based on discussions here in meta. Sometimes I edit if it's clear the commenter was trying to be helpful but may be misinterpreted, and I'll do this if there's been no follow up, just to help prevent any trouble resulting from the comment.
If someone is clearly trolling or trying to be offensive, I'll generally remove the comment.
As far as how many people are flagging, that's tough to answer. I don't always get to see who flagged. Comment flags aren't generally as critical as flags on posts, so I believe it only shows the flagged comment and the reason.
Finally, I encourage you to continue flagging. It's a way to draw attention to trouble-spots on the site. We should check MSO to see if dismissed comment flags affect the elusive flag weight. I don't think they do, but we should check.