The Bible calls idolatry idolatry, and it calls greed idolatry.
If someone is bowing down to idols, participating in rituals to worship spirits through idols, etc. you can call that idolatry.
If someone likes football too much, you shouldn't just assume it is idolatry and accuse him of one of the worst sins. If someone is has some wrong doctrine, that doesn't make him or her an idolater.
If your mom stole a pen from the bank, wouldn't it be wrong for someone to call your mother a whore... for stealing a pen? If they shouldn't call your mother a whore for that, then you shouldn't call people idolaters for things that aren't idolatry.
I think the broad definition of idolatry is derived from preaching that the 10 commandments say 'Thou shalt not have any other god before Me.' But think of 'before Me' as in his presence or something along that line. God wasn't saying it's okay to have a second god as long as it is second in rank.
A lot of westerners, not familiar with actual literal idolatry are more concerned with their various metaphorical idolatries than with the literal thing. These weren't westerners, but was in the home of some church going folks and they thought nothing of having a Buddha statue on the wall. Some people think of that as art, but what if you have a Buddhist guest. And art or no, we shouldn't have such things sitting on our shelves. Another guy thought it was no big deal to burn incense to Buddha but seemed concerned over metaphorical idolatry.