Go to the part where Ben Stiller is at the airport gate. Absolutely NO ONE is at the gate except for him. When the attendant calls for boarders, no one comes. But then, when Ben is on the plane, trying to stow his bag, the plane is full!! They couldn't have come before Ben, because the attendant called the whole plane, and no one came!
Probably he was late boarding. They often call row by row, starting with the back of the plane, before they call for the whole plane. Most people board row by row, and only the late comers board after that, which would explain why everyone was already on the plane.
You are right. If I am not mistaken they call each row while he is waiting and then he is able to bored. If he was late they would not be calling rows.