When Bourne is on the boat looking at a map and tying to figure who he is look at his sweater is backwards. Not only cant he remember who he is he cant remember how to dress.
Remember, he was on a fishing boat. Those boats are out at sea for weeks or even months at a time. There's no laundry facilities on a fishing boat. So it's not uncommon for fishermen to turn their clothes inside out after several days of wearing it so it "lasts longer."