If you actually listen to the DVD commentary, this issue is discussed. It is explained that we never actually see Cole's mom speaking to or reacting to David's presence.
When Cole comes into the room and sees his mom sitting across from David, what is mom actually is doing is sitting silently and thinking, while waiting for Cole to return home.
It is the fact that we get the sense that and ASSUME that Cole's mom has been talking with David that makes you THINK she can see him.
If you watch closely you will notice that they are both (Cole's mom and Malcolm) just sitting down. It gives the impression that they were talking, but we never really see it happen, then she just gets up and leaves.