This has been talked about before too.[1] I think the issue is generally that downmodding means different things to different people. Some use it to kill trolls, others to agree/disagree. I generally agree with you regarding how I downmod.

[1] http://news.ycombinator.com/item?id=147098