Newer stuff is always more expensive... It is always important to consider inflation as well. Think about the Nikon 600 f/4L VR, it was introduced in 2007 at around ~$9800. In 2012 dollars that comes out to $10,933.18 which is still less than the Canon lens, but much closer. Add in some of the advantages of the Canon model such as lighter weight and this makes sense.
Here are some other examples.
Look at the Nikon 70-200 f/2.8 VR (version 1), it usually sold for around $1699 and was introduced in 2003. Take inflation into account and you get $2,135.91. Again slightly under both new 70-200 offerings from both Canon and Nikon, but pretty darn close.
The Canon 24-70 was introduced in 2002 for about 220,000 yen. At the time ~125 yen was equal to 1 USD, making the lens $1750. Once again in 2012 dollars that is $2,250.17, just about on par with the price of the new 24-70LII. Be happy you aren't paying $3000 for it, which would be a more accurate price in terms of what the dollar is worth against the yen now.
Let's be conservative and say the Canon 35mm f/1.4L sold for $1200 in 1998 when it was released (It may have been more or perhaps less, some insight on this would be nice), well the new Nikon 35 1.4G retails for about $1650... What do you get when you adjust $1200 in 1998 to 2012 dollars? $1,702.95
Neither brand is really more expensive, the newer products from both manufacturers tend to be more pricey. The biggest reason isn't R&D or some crazy new technology either.
The USD is just worth less and less as the years go by.
Addendum: Think about your grocery bills. You are paying more now, but the food isn't making your body healthier. Inflation is the pits. Gas prices which have been manipulated over the years also impact food prices, so there won't be a totally direct correlation like there is in camera gear... heh