At the beginning of each show Drew Carey says points will be distributed. "The winner will do something special with me (Drew Carey) while the loser gets to [insert gag here]."
At the end of the show a ho-down is usually done. But if you notice the "winner" gets to sit at Drew's desk while the "loser(s)" get to do something with Drew.
Apparently the winner gets to watch while the losers do something with Drew.