They quip at each other and he establishes that she shares a similar past as an orphan. Vesper is introduced on a train headed to Montenegro with James Bond. And since your first thought about me ran to 'orphan', that's what I'd say you are.' ― Vesper Lynd to James Bond
Which means you were at that school by the grace of someone else's charity: hence that chip on your shoulder. But you wear it with such disdain, my guess is you didn't come from money, and your school friends never let you forget it. Naturally you think human beings dress like that. by the cut of your suit, you went to Oxford or wherever.