Upvoting and downvoting are not really part of moderation (at least not in the same way closing and reopening are) and are not the deciding factors of what belongs on the site. There's a reason why they're entirely distinct from closing and reopening.
You can start upvoting at 15 reputation, which is a pretty low bar. It's not too difficult to get that with a single question or answer, at which point you may have no idea what belongs on the site and what doesn't. You may even have gotten that reputation through the 100 reputation association bonus by coming from another site without having had any activity whatsoever on this site.
You need a whole 3000 reputation to cast a close vote (you can flag it before that, but flags can't close a question). That requires a bit more effort and hopefully the activity on the site to get there would've taught you what belongs here and/or you care enough to find out.
So yes, take the score of a post into account, because a high score may mean you're wrong in thinking it should be closed, or a negative score may mean you're wrong in thinking it should be open. But don't refrain from casting a close or reopen vote you're sure about only because of the score of the post.
If questions regularly manage to get high scores before getting closed, it may mean (a) we don't have enough moderators to deal with the volume or (b) there's disagreement over what should and shouldn't be closed among moderators.
We don't have that many questions, but whether we have too many voters that manage to spike up the score before it can get closed is an open question.
But there is definitely some fundamental disagreements about what should and shouldn't be closed between significantly sized groups of moderators on this site, which is a huge problem.
Note: by "moderators" I mean official and unofficial ones, who are able to cast close votes, i.e. anyone with more than 3000 reputation.
The bar for posting an answer is even lower, but answers have even less to do with moderation than upvoting or downvoting.
Although having too many answers may in fact indicate that a question is too broad.
Also keep in mind that a high score may indicate a question has potential, but it's missing something before it is a good fit for the site or it can be properly answered (meaning it should be closed and then reopened after being edited to fix any problems).