You will notice when Woody tries to get to Andy's house from Sid's window, the window is open. But after Sid complains about the storm shortly after, the window is closed because he bangs his head against it.
The window you talk about is not just one window, there are two windows. One, the one Woody talks to the other toys from is above a table, and the one Sid bangs his head on is just on a wall, not above a window.