A caste based society is a unique thing that we haven't seen in over 500 years and is almost an impossible thing to break the habit of.
In Jewish life there were the "12 tribes" (actually 13 but that's another discussion)
If you were a member of one of the 13 tribes you had a community that decided your fate and lot in life. You never had feelings of rejection or not being a part because you had a place in society at all times. If you were the "baker" or the inn keeper you had a definite role in society. Your "brother" Israeli would always protect you...they might be angry at you but you were always a brother who was needed to be a part of the community.
So long as you fulfilled your role as your father taught you to do things...your place in your community was assured. Of course you came to the "rescue " of other Israeli when needed by utilizing their businesses, farm produce, or products. They were always first choice when deciding where to purchase goods and services. You would always choose Gentiles last and truly bemoan that you had to deal with a Goyim(you didn't know or trust) for something that you needed.
With this sort of society you were constantly being told who you were, what you were, and subsequently that you belonged. "We need you" and "You are valuable" was the whispered tone of everyone you spoke with.
So subsequently you conformed within the parameters of Jewish life and lifestyle as a matter of habit. You simply didn't think of doing anything outside those parameters...it didn't really even come across your radar screen as being even the slightest bit wrong...in fact just the opposite. It was your duty to do these things because of the Law and the Prophets.
Which brings up the other half....
We tend to think of "worship" as something we take part in on a confined basis....where we meet and listen, sing and pray with others.
"Going to worship services" is always said by just about everyone I know....I'm sure everyone understands the expression. But Jews don't say that....ever!
God exists in and out of Time itself. God created time. These two thoughts are Jewish and true. However Jews believe that Abraham's relationship with God is crucial. They personally couldn't go to God to form their own relationship...and if they wished to share in Abraham's infinite reward they needed to be recognized as a descendant of Abraham and continue the same worship of God that he started. So everything that you did was worship of God. From the clothes you wore, the things you said, the food you ate, and the holidays you observed. Your whole life was an ongoing worship of God. You didn't even dream of starting a new worship of God because it would be so small that it would be ignored as a speck of dust by comparison of the larger worship that spanned hundreds of years. Your worship would be ignored if it wasn't offensive...which it likely would be.
So...this is the reason Peter had so many issues.