Yep, that's definitely how it works

Downvoting a post can decrease pending rewards and make it less visible. Common reasons:

• Disagreement on rewards
• Fraud or plagiarism
• Hate speech or trolling
• Miscategorized content or spam