Two threads on the Webmaster.com one started on March 18 2008 and another one is on May 20 2009.

First one says if your server is faster and the connection used is also faster the chance to increase the income around 25%. because this will increase the quality score. Read the Google Adwords Blog.

Here is the thread link

Now the second one it says if the server is slow or your page load time is high it will increase the CTR rate (means more money). In this nothing related to Google AdSense actually, a user who is impatient for the content to be loaded will click on advertisements. The negative side of this is a user may not be interested in a slow site.

Read the thread here

[Via Search Engine Roundtable]