The 40D was my first DSLR, and looking back at the photo's I took with it, it was a great camera! The 50D when it came out was of no interest to me - the 40D beats it in IQ.
After I got my 5D2 I sold the 40D, then started missing it for the crop factor. I then got a 7D and haven't really looked back at the 40D. Although... I did then buy a 400D for light travel use. The 400D is reminiscent of the 40D in that it uses the same sort of sensor (but with only Digic II, makes less good use of it).
Owning the 400D now is an eye opener, it's a convenient little camera but I am very happy I went with the more expensive 40D at the time.