You find it offensive because you are not familiar with it's background. Let me help you.
whenever some problem arise in muslim world they say's "Ye Israel ki saazish hay".
Similarly whenever some problem rises in Pakistan and Posted in this forum , some Pakistani member's used to say that "Ye Hindu banya ki saazish hay" . from that On it become a "funny" statement used by Indian member's to tease Pakistani member's.
Now a days even many Indian member's take it as "funny" . ask any Indian member.
BTW Banya in urdu is a person who do business of Fabric/textile items. i.e; suit's etc. so Logically speaking even my father is a bunya. and so am i.