Is it really that bad leaving contacts in all night?

I wear contacts and sometimes I'm just too tired to take them out before I go to bed. I know that's bad but, really how bad can it be. Will leaving them in over night cause damage to me eyes, if so how much? Can this ruin my eyes or not?