Sony Announces Addition of Uncompressed 14-Bit RAW Still Image Capture for New A Cameras

Mar 2, 2012
3,188
543
jrista said:
Mt Spokane Photography said:
jrista said:
This is totally incorrect. Photoshop tells you the uncompressed size for RGB DATA!!!!

RAW images store a SINGLE color value per PIXEL. Pixel being directly correlated to the physical element on the sensor. When Photoshop opens an image, it generates an in-memory RGB-per-pixel bitmap. If you loaded the image as 8-bpp, then that is a total of 24-bits per pixel, rather than 14-bits. When you account for the compression, THAT is where you get the 2:1 to 3:1 "ratio".

Same thing goes for saving an image as TIFF. The TIFF image is again RGB, so three color values per pixel. Throw in the increase to 16-bit, and you are radically inflating the actual necessary data sizes here.

I came up with an ~10mb difference by calculating the actual necessary data storage to contain ONE 14-bit data value per sensor pixel, and compared that to actual file sizes. I just looked them up online. As I said, it's very simple math.

Your simple math does not line up with reality. The early test files have doubled in size.

Figures are available already for Nikon models that offer lossy compression, lossless compression, and uncompressed. They also run slightly greater than 2:1.

Scroll down to the section listing typical card capacities where it gives expected file sizes.

http://imaging.nikon.com/lineup/dslr/d4/spec.htm


So, ignoring the masked border pixels, the sensor size is 4,928x3,280. Multiplying those together give us the total pixel count of 16,163,840. That is the exact number of light sensitive pixels. To store all of them with UNCOMPRESSED 14-bit precision, mathematically, you need 16,163,840 * 14 BITS, or a total of 226,293,760. Since there are 8 bits per byte, we divide that by 8 to get the number of BYTES, which comes out to 28,286,720. An uncompressed 14-bit RAW image form a sensor with 4,928x3,280 pixels will consume 28.3MB worth of space if it is stored at full precision, uncompressed.

Now, the page you linked says an uncompressed 14-bit NEF for the D4 is 15.3 MB.


Are we looking at the same page?

My D4 spec page says 34.3MB for 14-bit uncompressed D4 files, which is pretty well in line with the bit arithmetic + a JPEG preview and overhead data.

I don't know where you're seeing 15.3MB. Perhaps you misread the 12-bit compressed figure?
 

Attachments

  • Screen Shot 2015-09-24 at 7.29.20 PM.png
    Screen Shot 2015-09-24 at 7.29.20 PM.png
    58.6 KB · Views: 236
Upvote 0

Don Haines

Beware of cats with laser eyes!
Jun 4, 2012
8,246
1,939
Canada
jrista said:
Mt Spokane Photography said:
jrista said:
This is totally incorrect. Photoshop tells you the uncompressed size for RGB DATA!!!!

RAW images store a SINGLE color value per PIXEL. Pixel being directly correlated to the physical element on the sensor. When Photoshop opens an image, it generates an in-memory RGB-per-pixel bitmap. If you loaded the image as 8-bpp, then that is a total of 24-bits per pixel, rather than 14-bits. When you account for the compression, THAT is where you get the 2:1 to 3:1 "ratio".

Same thing goes for saving an image as TIFF. The TIFF image is again RGB, so three color values per pixel. Throw in the increase to 16-bit, and you are radically inflating the actual necessary data sizes here.

I came up with an ~10mb difference by calculating the actual necessary data storage to contain ONE 14-bit data value per sensor pixel, and compared that to actual file sizes. I just looked them up online. As I said, it's very simple math.

Your simple math does not line up with reality. The early test files have doubled in size.

Figures are available already for Nikon models that offer lossy compression, lossless compression, and uncompressed. They also run slightly greater than 2:1.

Scroll down to the section listing typical card capacities where it gives expected file sizes.

http://imaging.nikon.com/lineup/dslr/d4/spec.htm

Those file sizes are illogical. This is really basic math. Excluding the masked border pixels, which would increase file size, this is a matter of bits and bytes. The sensor in the D4 is a bayer sensor. That means there is a header in the NEF that defines the bayer pattern, among other bits of metadata, and that is probably only a matter of bytes in size, so a minuscule fraction of the data. The rest of the data is pixel information. Every pixel is stored in a sequence with a specified run length, as a single value of the specified camera precision (in the case of the D4, apparently either 12 bits or 14 bits), until the last pixel is written.

So, ignoring the masked border pixels, the sensor size is 4,928x3,280. Multiplying those together give us the total pixel count of 16,163,840. That is the exact number of light sensitive pixels. To store all of them with UNCOMPRESSED 14-bit precision, mathematically, you need 16,163,840 * 14 BITS, or a total of 226,293,760. Since there are 8 bits per byte, we divide that by 8 to get the number of BYTES, which comes out to 28,286,720. An uncompressed 14-bit RAW image form a sensor with 4,928x3,280 pixels will consume 28.3MB worth of space if it is stored at full precision, uncompressed.

Now, the page you linked says an uncompressed 14-bit NEF for the D4 is 15.3 MB. I'm sorry, but that plain and simply cannot be the case. Something has to give in order for that to be true. Either those files are actually still compressed in some way, or Nikon is actually gimping you on the bit depth...or, the number they are reporting for uncompressed 14-bit RAW is simply wrong. I am not inclined to think Nikon is gimping their photographers in terms of precision, so they are either still compressing those images, or the numbers are wrong. The numbers don't add up in any way that I can account for...so I'm inclined to say the numbers, all of them, for the D4, are just wrong.

I trust the math, more than some spec page on the internet, even if it is official.

I've read around the net that the average image file sizes for the 5Ds range from low 60's to low/mid 70's. With an image size of 8688x5792, it's uncompressed file size would be 88MB. At 62mb, the image would be 70.5% of the full uncompressed size, and at 74mb the image would be 84.1% the full uncompressed size. That is a top savings of less than 30%. Maximum savings would be around 26mb, minimum savings would be around 14mb.

In the case of my 5D III, which is the key source where I actually pulled the ~10mb difference from, my uncompressed size is ~39MB, and my actual sizes in practice range from 27MB to 32MB. At 27MB, I'm saving ~30%, at 32MB I am saving ~18%.

Simple math. Assuming, of course, that were really talking about totally uncompressed, full precision RAW data. God only knows what Nikon is reporting on the D4 page, but it isn't uncompressed full precision RAW file sizes. As for the in-memory load when opening an image in PS, that is the RGB (three full precision values per pixel) load. In the case of a 5Ds, with it's 8688x5792 pixel images, at 8-bpp, the in-memory load should be somewhere around 150,962,688 bytes (~150MB). If we are talking 16-bpp, then that jumps to 301,925,376 bytes (~300MB). If you look around the net, people are seeing 145-150mb loads for 8-bpp and right around 300mb loads for 16-bpp with 5Ds images loaded into Photoshop.

Simple math here.
Today's shooting 465 images. Size is 11,352,034,319 bytes.
Largest image is 31,101,952 bytes
smallest image is 19,260,416 bytes
Average file size of 24,412,977 bytes

The image size is 5472 x 3648, or 19,961,856 pixels
at 14 bit depth that would be 34,933,248 bytes

My minimum compression is 10.97 percent
my maximum compression is 44.86 percent
my average compression is 30.11 percent

My 7D2 aligns quite well with your ~30 percent number from a 5DIII

There are a lot of compression algorithms out there.... most seem to be public domain, which means that anyone can use them. If the best that Canon seems to able to do for lossless compression is 30 percent, then it is a fair bit that the best Nikon or Sony will be able to do is also to AVBERAGE 30 percent.

A claim of lossless compression where there is a 50 percent reduction in file size could be a mistake, a lie, or it could be the marketing people using a best case scenario and not an average or a typical value. Remember, on my 7D2 today I managed to take one image that got compressed 44.86 percent....

I believe that the claim of 50 percent reduction really is for lossless compression, but is a best case scenario
 
Upvote 0
Mar 2, 2012
3,188
543
Don Haines said:
A claim of lossless compression where there is a 50 percent reduction in file size could be a mistake, a lie, or it could be the marketing people using a best case scenario and not an average or a typical value.


I wondered myself whether it was based on dramatically under- or over-exposed images, wherein an abundance of repetitive 1s or 0s would lend itself to high compression.
 
Upvote 0
3kramd5 said:
Don Haines said:
A claim of lossless compression where there is a 50 percent reduction in file size could be a mistake, a lie, or it could be the marketing people using a best case scenario and not an average or a typical value.


I wondered myself whether it was based on dramatically under- or over-exposed images, wherein an abundance of repetitive 1s or 0s would lend itself to high compression.

Yeah, that is a possibility. Not indicative of real-world cases, in which case, it's very misleading. The more complex the data is, the more detailed the scene (or, the more noisy the image), the less your compression ratio is likely going to be. I've never seen more than about 30% with any of my Canon cameras, and it is usually more around 20%. Maybe Sony could do better with lossless compression...I am honestly not sure how powerful their Bionz chips are.

It also sounds more and more like there are embedded ARM processors going into cameras. With more general purpose high speed compute power, better algorithms would be applicable as well.
 
Upvote 0
3kramd5 said:
Are we looking at the same page?

My D4 spec page says 34.3MB for 14-bit uncompressed D4 files, which is pretty well in line with the bit arithmetic + a JPEG preview and overhead data.

I don't know where you're seeing 15.3MB. Perhaps you misread the 12-bit compressed figure?

Hmm...strange. I just clicked the link Spokane provided and scrolled down.
 
Upvote 0
jrista said:
3kramd5 said:
Are we looking at the same page?

My D4 spec page says 34.3MB for 14-bit uncompressed D4 files, which is pretty well in line with the bit arithmetic + a JPEG preview and overhead data.

I don't know where you're seeing 15.3MB. Perhaps you misread the 12-bit compressed figure?

Hmm...strange. I just clicked the link Spokane provided and scrolled down.

Aha. Apparently, I scrolled too far. Scrolling back up a bit, there is another table. I guess I was looking at DX crop sizes.
 
Upvote 0

Don Haines

Beware of cats with laser eyes!
Jun 4, 2012
8,246
1,939
Canada
jrista said:
3kramd5 said:
Don Haines said:
A claim of lossless compression where there is a 50 percent reduction in file size could be a mistake, a lie, or it could be the marketing people using a best case scenario and not an average or a typical value.


I wondered myself whether it was based on dramatically under- or over-exposed images, wherein an abundance of repetitive 1s or 0s would lend itself to high compression.

Yeah, that is a possibility. Not indicative of real-world cases, in which case, it's very misleading. The more complex the data is, the more detailed the scene (or, the more noisy the image), the less your compression ratio is likely going to be. I've never seen more than about 30% with any of my Canon cameras, and it is usually more around 20%. Maybe Sony could do better with lossless compression...I am honestly not sure how powerful their Bionz chips are.

It also sounds more and more like there are embedded ARM processors going into cameras. With more general purpose high speed compute power, better algorithms would be applicable as well.
I shot off 465 today and averaged 30 percent..... Some were as low as 10 percent, but several shots of the moon compressed 40+ percent..... Lots and lots of black in the images.....

Try compressing a Cr2 file on your home computer..... Will it compress further?
 
Upvote 0
Jul 21, 2010
31,200
13,071
jrista said:
3kramd5 said:
Are we looking at the same page?

My D4 spec page says 34.3MB for 14-bit uncompressed D4 files, which is pretty well in line with the bit arithmetic + a JPEG preview and overhead data.

I don't know where you're seeing 15.3MB. Perhaps you misread the 12-bit compressed figure?

Hmm...strange. I just clicked the link Spokane provided and scrolled down.

You read the value for the DX (1.5x crop) image.
 
Upvote 0
Nov 4, 2011
3,165
0
3kramd5 said:
There you have it: (less than) half the buffer.

https://youtu.be/ykXCQnzXm-8

As expected. Miracles are rare in physics, hehe!
In practice it does not matter much. Situations were full speed / continuous fire is needed, are usually different from those were maximum image quality is needed. Users now have choice, speed vs. Quality (and corresponding file size) - which is fine with me. Maybe Sony "re-invents" lossless compression in A7 series Mark III. Those who need it, will happily shell out 3500 Euro in order to get it. All others will just shrug and laugh. No big deal.

I'd happily buy the Sony A7/R II specs in a Canon mirrorless system. But ... there is none.
 
Upvote 0
Mar 2, 2012
3,188
543
AvTvM said:
3kramd5 said:
There you have it: (less than) half the buffer.

https://youtu.be/ykXCQnzXm-8

As expected. Miracles are rate in physics, hehe!
In practice it does not matter much. Situations were full speed / continuous fire is needed, are usually different from those were maximum image quality is needed.

I personally always want maximum image quality. :p

Of course, with the Sony, when you enable continuous, it drops precision*, so maximum image quality isn't available.

*I haven't confirmed it still does this with uncompressed raws.
 
Upvote 0
Nov 17, 2011
5,514
17
3kramd5 said:
AvTvM said:
3kramd5 said:
There you have it: (less than) half the buffer.

https://youtu.be/ykXCQnzXm-8

As expected. Miracles are rate in physics, hehe!
In practice it does not matter much. Situations were full speed / continuous fire is needed, are usually different from those were maximum image quality is needed.

I personally always want maximum image quality. :p

Of course, with the Sony, when you enable continuous, it drops precision*, so maximum image quality isn't available.

*I haven't confirmed it still does this with uncompressed raws.

Haven't update my a7r II yet. Let's us know how you like it :)
 
Upvote 0
Mar 2, 2012
3,188
543
Dylan777 said:
Haven't update my a7r II yet. Let's us know how you like it :)

I don't really see a difference (note: I have not shot star tracks or anything like that in an attempt to illicit artifacting).

The theory (propaganda) behind sony's compression was that the primary loss was to noise. Maybe that was somewhat true.
 
Upvote 0
Nov 17, 2011
5,514
17
3kramd5 said:
Dylan777 said:
Haven't update my a7r II yet. Let's us know how you like it :)

I don't really see a difference (note: I have not shot star tracks or anything like that in an attempt to illicit artifacting).

The theory (propaganda) behind sony's compression was that the primary loss was to noise. Maybe that was somewhat true.

Thanks for feedbacks. I'm a bit concern loading 80MP files to my PC.
 
Upvote 0
Mar 2, 2012
3,188
543
There too I don't particularly notice a difference, perhaps because I do other stuff while I import. Working with the files feels the same, probably because regardless of how the data is stored, it is uncompressed in memory during processing (so a compressed file is just as "heavy" as an uncompressed one once loaded by camera raw or whatever you fancy).
 
Upvote 0
Feb 28, 2013
1,616
281
70
Dylan777 said:
3kramd5 said:
Dylan777 said:
Haven't update my a7r II yet. Let's us know how you like it :)

I don't really see a difference (note: I have not shot star tracks or anything like that in an attempt to illicit artifacting).

The theory (propaganda) behind sony's compression was that the primary loss was to noise. Maybe that was somewhat true.

Thanks for feedbacks. I'm a bit concern loading 80MP files to my PC.
Im not loading them to my PC from the 5DS but to an external drive specifically to manage the larger file sizes. Orginals and edited shots are on the same drive.
 
Upvote 0