Help! 1DX II Cfast issues!

[Halfrack]

I do not let any software do the copying of my files from CF or SD cards (and Cfast in a few days!) onto computer HDs. I do it always with the OS.

Part of why PhotoMechanic is so amazing is it allows you to build ingest processes and set multiple destinations with multiple sources. It also allows you to add IPTC info to everything on the way in.

Importing via an iPhoto/Photos/Lightroom/Aperture/etc process where it stuffs your photos into it's own file structure is bad and where a lot of folks have issues.

It sounds like you do not understand Lightroom at all. It puts photos where you tell it in your own file structure. Lightroom will not delete photos from the card either.


What Lightroom does do is to create a database with a list of the photos and the location where I put them. It never ever changes the originals. You can change the settings to have it create sidecar files with the data in them and stored with the images. Twice the number of files greatly increases the possibility of a glitch, but you can always re-edit your 100,000 files.

Its a very secure and reliable way of handling photos. I don't use aperture, but I doubt that it stuffs photos into its own file structure either.

Photomechanic is not a photo editor, or at best a weak one. Its practical use is for key wording files. It adds the information into a sidecar which is easy to lose and get disconnected from the original file. You can always search for it or re-edit your 100,000 files if that happens though.

I'm fully aware of how Lightroom handles imports, haven't liked it since v3, and iPhoto/Photos/Aperture are much worse. Raise your hand if you've dealt with recovering photos out of a Library that isn't fixed by a rebuild. 'Referenced' should be the only way photos are handled, 'Managed' should be dragged out back and shot. All the Apple products pushed the 'managed' method early on.

Lightroom CC/6 will delete files if you select 'move' for import, and it'll prompt you after import if you 'copy' or 'copy as dng' if you want to delete the source files. 'Add' is the method I've used forever.

Sidecar files are fine, and are required when moving between different software packages.

The intent was to utilize a method of copying files off cards on to a computer in a way that doesn't try to read/preview the data that it may not understand due to file format being newer than the raw convertor.

 

[fish_shooter]

I do not let any software do the copying of my files from CF or SD cards (and Cfast in a few days!) onto computer HDs. I do it always with the OS.

Tell me, what OS are you using that isn't software?

Roger that, the OS is software. One of the above posts in this threads explains very well why the OS is better than software layered on top of the OS. I should add - I remove the CF card from the computer before starting any image editing as another level of insurance.

 

[fish_shooter]

I do not let any software do the copying of my files from CF or SD cards (and Cfast in a few days!) onto computer HDs. I do it always with the OS.

Part of why PhotoMechanic is so amazing is it allows you to build ingest processes and set multiple destinations with multiple sources. It also allows you to add IPTC info to everything on the way in.

Importing via an iPhoto/Photos/Lightroom/Aperture/etc process where it stuffs your photos into it's own file structure is bad and where a lot of folks have issues.

It sounds like you do not understand Lightroom at all. It puts photos where you tell it in your own file structure. Lightroom will not delete photos from the card either.


What Lightroom does do is to create a database with a list of the photos and the location where I put them. It never ever changes the originals. You can change the settings to have it create sidecar files with the data in them and stored with the images. Twice the number of files greatly increases the possibility of a glitch, but you can always re-edit your 100,000 files.

Its a very secure and reliable way of handling photos. I don't use aperture, but I doubt that it stuffs photos into its own file structure either.

Photomechanic is not a photo editor, or at best a weak one. Its practical use is for key wording files. It adds the information into a sidecar which is easy to lose and get disconnected from the original file. You can always search for it or re-edit your 100,000 files if that happens though.

1st - Here is a (somewhat old but still relevant) thread about "Delete photos after Import" discussion... https://forums.adobe.com/message/1434331#1434331#1434331

2nd - I almost purchased PhotoMechanic a year or two ago but after a bad experience with the individuals that run that company, I was motivated to look for other options. For what PhotoMechanic does (and does pretty well), it is waaay over priced. I found a very good (actually BETTER) alternative in FastRawViewer. - http://www.fastrawviewer.com/ Thank goodness that the PhotoMechanic folks pissed me off because they saved me a lot of money and helped me find a BETTER SOLUTION for what I needed. PhotoMechanic is fine but I think it was much more necc and relevant about 5 years ago. Now I think it should cost about $25 for what it does in today's more robust software ecosystem.

3rd - Lightroom does have the tendency/ability to allow folks to just let everything dump into a "big bucket" of files and let the LR catalog sort it all out for you. I DON'T do it that way but this is one way people tend to use Lightroom.

#3 is what I do - it fits well with what I do in photography. I shoot mainly under water and mainly salmon. I have tens of thousands of pix of some of the species taken over multiple years. My LR library is over 400K images and takes up three 6-TB HDs. With LR I can extract pix of salmon doing various things since it is all key worded. This useful for finding an image to show or deliver to someone. Also I use the images as data when planning photo trips since salmon do more or less the same thing each year on a similar schedule. I can also round up all the pix taken of a given species (non-salmon, far few pix per sp.) (or other key-worded trait) - I just did this having just returned from HI - and evaluate them (such as which ones to put on my web site), verify ID, etc.

Other branches of photography may be much more specific job oriented (e.g. a wedding) and the images can be discarded or archived away after the job is done. The need for an extensive database is not as great. Maybe just a few select images are kept for portfolio.

 

[romanr74]

I do not let any software do the copying of my files from CF or SD cards (and Cfast in a few days!) onto computer HDs. I do it always with the OS.

Part of why PhotoMechanic is so amazing is it allows you to build ingest processes and set multiple destinations with multiple sources. It also allows you to add IPTC info to everything on the way in.

Importing via an iPhoto/Photos/Lightroom/Aperture/etc process where it stuffs your photos into it's own file structure is bad and where a lot of folks have issues.

It sounds like you do not understand Lightroom at all. It puts photos where you tell it in your own file structure. Lightroom will not delete photos from the card either.


What Lightroom does do is to create a database with a list of the photos and the location where I put them. It never ever changes the originals. You can change the settings to have it create sidecar files with the data in them and stored with the images. Twice the number of files greatly increases the possibility of a glitch, but you can always re-edit your 100,000 files.

Its a very secure and reliable way of handling photos. I don't use aperture, but I doubt that it stuffs photos into its own file structure either.

Photomechanic is not a photo editor, or at best a weak one. Its practical use is for key wording files. It adds the information into a sidecar which is easy to lose and get disconnected from the original file. You can always search for it or re-edit your 100,000 files if that happens though.

+1

 

[YuengLinger]

I want to add one other habit I try to use...

Multiple CF Cards. Especially if I'm travelling or it's a big shoot or a very important shoot. I would rather have 4 x 16 GB cards than one 64 GB card. Most of my cards are no larger than 32 GB and I very rarely fill them up.

- I'll swap cards after an hour or two of an important shoot.
- I'll swap cards at least every day while travelling (depending pictures value and number of shots).
- If I travel for more than a couple days, I take my Nexto media drive "backer-upper".

Why? If one card is failing, I only lose a portion of the images. If the camera is lost or stolen, I only lose a portion of the images. In other words, keep the images in multiple locations, even when those locations are your pockets, the camera, the backpack, another drive, etc. Keeping all of your images on one large media is risky, hazardous and honestly... lazy. I've done it before on afternoon shoots where I shoot a lot and I'll be back in a couple hours. But even then I know I'm risking image loss if something bad happens.

I use blue painter's tape on all my CF cards with a tab hanging off the back. When I pull a card from the camera, I move the tape around to cover the pin hole side and I know it's a "used" card.

While swapping out cards like this makes some sense, and is working for you, I remember Arthur Morris said during a workshop that the only time he lost images on a CF card was when he was swapping out and dropped one in a muddy pond.

His point was that the odds of losing cards by worrying about how many images have been recorded has to be balanced against the very low failure rates of top brand cards.

All about balance and getting stuff backed up ASAP but without losing great shots because you are changing cards too often.

 

[RustyTheGeek]

I want to add one other habit I try to use...

Multiple CF Cards. Especially if I'm travelling or it's a big shoot or a very important shoot. I would rather have 4 x 16 GB cards than one 64 GB card. Most of my cards are no larger than 32 GB and I very rarely fill them up.

- I'll swap cards after an hour or two of an important shoot.
- I'll swap cards at least every day while travelling (depending pictures value and number of shots).
- If I travel for more than a couple days, I take my Nexto media drive "backer-upper".

Why? If one card is failing, I only lose a portion of the images. If the camera is lost or stolen, I only lose a portion of the images. In other words, keep the images in multiple locations, even when those locations are your pockets, the camera, the backpack, another drive, etc. Keeping all of your images on one large media is risky, hazardous and honestly... lazy. I've done it before on afternoon shoots where I shoot a lot and I'll be back in a couple hours. But even then I know I'm risking image loss if something bad happens.

I use blue painter's tape on all my CF cards with a tab hanging off the back. When I pull a card from the camera, I move the tape around to cover the pin hole side and I know it's a "used" card.

While swapping out cards like this makes some sense, and is working for you, I remember Arthur Morris said during a workshop that the only time he lost images on a CF card was when he was swapping out and dropped one in a muddy pond.

His point was that the odds of losing cards by worrying about how many images have been recorded has to be balanced against the very low failure rates of top brand cards.

All about balance and getting stuff backed up ASAP but without losing great shots because you are changing cards too often.

Mistakes can happen, that's a fact. My point isn't a hard and fast rule where I change a card at X number of images in a hurry with my hands full in the middle of an event. It's simply a guideline where I want to minimize loss if a card's images are compromised from a variety of possible errors. It could be electronic, external physical or loss/theft. Whatever the case, if a portion of the images are on another card kept safely outside of the camera, that's still better than the entire set being lost with the camera.

 

[HKFEVER]

Just use Mac's Terminal and ran MD5 the result for the _X2_0431.cr2 are:
good CR2 file with 6b1873ddcb8268b3739436925d4d4ce5
bad CR2 file with f2dbdce652c274f3b66838b54a2dfb28

Good file is like this after PS conversion:

Bad file is like this after PS conversion:

Bad file is like this after DPP4 conversion:

 

[JMZawodny]

Just use Mac's Terminal and ran MD5 the result for the _X2_0431.cr2 are:
good CR2 file with 6b1873ddcb8268b3739436925d4d4ce5
bad CR2 file with f2dbdce652c274f3b66838b54a2dfb28

What does an MD5 checksum have to do with anything? Certainly you are not suggesting that all "good" files have the same MD5 string.

 

[scottkinfw]

I'm really sorry to hear your story. It truly sucks. :'(

This is why I typically wait a while before I buy the newest tech (camera or otherwise) and then I test the crap out of it before I use it in "official" work. (ie: Work for others or "clients".) Plus, the 1DX-II has dual card slots for redundancy.

Please keep us posted on how things turn out and GOOD LUCK.

Do you think it's possibly a BAD CARD? Also, make sure it's not a COUNTERFEIT CARD! I always test every card before I put them in service to avoid these types of problems. Even legitimate memory can have problems occasionally.

https://sosfakeflash.wordpress.com/

Rusty, thanks for that great link. I'm a Mac guy and it doesn't run on Mc OS. Do you have recommendations for the Mac OS?

Thanks.

sek

 

[scottkinfw]

For my own workflow I do not let any software do the copying of my files from CF or SD cards (and Cfast in a few days!) onto computer HDs.

Also good advice. Manually copy the files from the card, don't 'import' them. After copying, import them from your local drive into your favorite editor/converter.

May I ask why? You guys have got me mildly worried I'm doing something wrong now, but I've never had a problem letting Lightroom import the photos for me... What could go wrong? :-\

To be safe, I create a folder on my backup storage device (I use a Drobo/RAID). Then I copy to the new file. Then I import to LR. Better, I "add" and let the file remain in place where it is safe and a catalogue is created in LR.

In answer to your question, I don't know everything that could go wrong, but in my mind, this takes a lot of the unknowns out of the equation while giving me control over the importing process and has never given me a problem-yet.

sek

 

[scottkinfw]

I want to add one other habit I try to use...

Multiple CF Cards. Especially if I'm travelling or it's a big shoot or a very important shoot. I would rather have 4 x 16 GB cards than one 64 GB card. Most of my cards are no larger than 32 GB and I very rarely fill them up.

- I'll swap cards after an hour or two of an important shoot.
- I'll swap cards at least every day while travelling (depending pictures value and number of shots).
- If I travel for more than a couple days, I take my Nexto media drive "backer-upper".

Why? If one card is failing, I only lose a portion of the images. If the camera is lost or stolen, I only lose a portion of the images. In other words, keep the images in multiple locations, even when those locations are your pockets, the camera, the backpack, another drive, etc. Keeping all of your images on one large media is risky, hazardous and honestly... lazy. I've done it before on afternoon shoots where I shoot a lot and I'll be back in a couple hours. But even then I know I'm risking image loss if something bad happens.

I use blue painter's tape on all my CF cards with a tab hanging off the back. When I pull a card from the camera, I move the tape around to cover the pin hole side and I know it's a "used" card.

I don't agree with that. Losing just one portion of a photoshoot could be devastating. Some things can't be redone or reproduced. I shoot people, not landscapes or birds. I can't afford to lose anything at all.
In 14 years, I only had 6 cards fail on me and lost everything. Only 6 times. Not terrible.
If your camera supports it, the best is to write to both of them simultaneously. That is a redundant real time backup.
I have done it many times specially after switching to Lexar, which has a habit of failing more than Sandisk...

Um... no offense but you're OK with SIX TIMES of LOST IMAGES !???! I'm glad you're honest but this only happened to me ONCE many years ago. That was enough to motivate me to get more disciplined and begin the more fault tolerant workflow I outlined in the posts above. Since then I've lost NOTHING and since I test my cards before I use them, I've returned (exchanged) 2 or 3 bad cards over the years before I lost even one image on them. I've also suffered failed hard drives with no lost images.

So my friendly feedback here is that based on the information you have provided you are using a system that is NOT particularly FAULT TOLERANT in several areas. You say the (multiple) errors are flukes and you still prefer that system. This isn't logical. You also say that your images are so important that you prefer to keep them all on one card (in lieu of spreading out the risk of loss). This isn't logical either. You say that importing the images via Aperture is easier and you prefer it even though it has burned you and others have expressed their more reliable methods. So I'm confused. Is there any part of your workflow you plan to change in order to avoid losing images or do you simply plan to hope it works better all by itself?

Please don't get mad and flame me. I'm not trying to insult you. I think I'm asking a legitimate question. I understand that old habits are hard to break but losing images is harder IMHO. But I'm just an old IT/Engineer type guy that values data redundancy.

Well said, cogent argument that can't be argued.

The old adage comes to mind, (paraphrasing) Repeating the same behavior that gets the same outcome every time with the expectation of a different outcome is the definition of insanity.

Time to make a change I would say. With respect, of course.

sek

 

[scottkinfw]

HINAP

HINAP ??? Never seen that one before. (Neither has Google.) I'm stumped!

Hope is not a plan.

Courtesy of Uncle Roger:
https://www.lensrentals.com/blog/2010/07/fwigtew-and-other-first-wedding-acronyms/

It is the plan of last resort, and certainly not for atheists.

sek

 

[HKFEVER]

Just use Mac's Terminal and ran MD5 the result for the _X2_0431.cr2 are:
good CR2 file with 6b1873ddcb8268b3739436925d4d4ce5
bad CR2 file with f2dbdce652c274f3b66838b54a2dfb28

What does an MD5 checksum have to do with anything? Certainly you are not suggesting that all "good" files have the same MD5 string.

I guess you are confused.

6b1873ddcb8268b3739436925d4d4ce5 was generated from good _X2_0431.cr2

f2dbdce652c274f3b66838b54a2dfb28 was generated from _X2_0431.cr2 after it got corrupted.

It clearly shows something has written something into the good _X2_0431.cr2 and corrupted it.

 

[zim]

Just use Mac's Terminal and ran MD5 the result for the _X2_0431.cr2 are:
good CR2 file with 6b1873ddcb8268b3739436925d4d4ce5
bad CR2 file with f2dbdce652c274f3b66838b54a2dfb28

Not sure about using checksum, I'd imagine that processing the same file twice with no errors could produce results with different checksums after all the file is being "processed".
However that test is interesting! The file ooc wasn't corrupt so in this case it's not a card fault. If you processed that file 10 times in PS how many were good uncorrupted images?

 

[Ozarker]

For my own workflow I do not let any software do the copying of my files from CF or SD cards (and Cfast in a few days!) onto computer HDs.

Also good advice. Manually copy the files from the card, don't 'import' them. After copying, import them from your local drive into your favorite editor/converter.

This is exactly what I have always done....

- Pull card from camera, use reliable reader.
- Copy everything to an organized directory on the "PicsB4LR" Drive.
- Use "FastRawViewer" to cull out unwanted images.
- Import in LR from the "PicsB4LR" drive to a PICTURES directory on a different RAID array drive.
- Store CF card safely until I finish processing, exporting and uploading images to Zenfolio.

Notice that there are images in 2, 3 and finally 4 places before I re-use the CF card.

Smart Rusty.

 

[HKFEVER]

Just use Mac's Terminal and ran MD5 the result for the _X2_0431.cr2 are:
good CR2 file with 6b1873ddcb8268b3739436925d4d4ce5
bad CR2 file with f2dbdce652c274f3b66838b54a2dfb28

Not sure about using checksum, I'd imagine that processing the same file twice with no errors could produce results with different checksums after all the file is being "processed".
However that test is interesting! The file ooc wasn't corrupt so in this case it's not a card fault. If you processed that file 10 times in PS how many were good uncorrupted images?

Those code won't change as long as the file remain the same.

Process Raw file with PS only create another file (the file format is based on your selection when you save the new created file). The process won't change the original Raw file.

The corrupted file is not from processing the raw file. You can see the corrupted file when your preview it in Br or double click it in DPP4.

If I processed 10 time with good raw file, all processed file are good.
If I processed 10 time with corrupted raw file, all processed file are bad.

The point is why some good raw file (in the CFast card) turn bad after while in hrs or days? At some point, computer, reader, card, or camera must read the CFast card and cause the corruption.

 

[zim]

Just use Mac's Terminal and ran MD5 the result for the _X2_0431.cr2 are:
good CR2 file with 6b1873ddcb8268b3739436925d4d4ce5
bad CR2 file with f2dbdce652c274f3b66838b54a2dfb28

Not sure about using checksum, I'd imagine that processing the same file twice with no errors could produce results with different checksums after all the file is being "processed".
However that test is interesting! The file ooc wasn't corrupt so in this case it's not a card fault. If you processed that file 10 times in PS how many were good uncorrupted images?

Those code won't change as long as the file remain the same.

Process Raw file with PS only create another file (the file format is based on your selection when you save the new created file). The process won't change the original Raw file.

The corrupted file is not from processing the raw file. You can see the corrupted file when your preview it in Br or double click it in DPP4.

If I processed 10 time with good raw file, all processed file are good.
If I processed 10 time with corrupted raw file, all processed file are bad.

The point is why some good raw file (in the CFast card) turn bad after while in hrs or days? At some point, computer, reader, card, or camera must read the CFast card and cause the corruption.

Ah right, your checksuming the same original unprocessed file twice.

The notion of a read only process on any file system causing a corruption on the original file is scary actually. Wonder if a binary copy to a temp folder first would stop that?

 

[tpatana]

I'm wondering why it only impacts that one section.

Thinking aloud (=mostly stupid/wrong guesses):

-Could they be at high ISO images mostly, and something clips the file size and corrupts beyond certain size, hence messing up the bottom right corner (assuming raw is written somewhat from top-left).

-I assume you checked the file sizes for the good and bad file, and they are same? If yes, can you run some bit-by-bit compare SW to check if the problem really starts at certain location. And maybe run it for few different similar corrupted files to see if the bits changes always at the same location

 

[HKFEVER]

The followings are the links for good raw files that I copied from CFast to Mac's desktop:
https://dl.dropboxusercontent.com/u/80110215/Good%20Raw%20Files/_X2_0502.CR2
https://dl.dropboxusercontent.com/u/80110215/Good%20Raw%20Files/_X2_0503.CR2
https://dl.dropboxusercontent.com/u/80110215/Good%20Raw%20Files/_X2_9844.CR2

The followings are the links for good raw files that turned corrupted in hrs or days in CFast (I kept all the original raw files in CFast) which were good previously in CFast:
https://dl.dropboxusercontent.com/u/80110215/Corrupted%20Raw%20Files/_X2_0502.CR2
https://dl.dropboxusercontent.com/u/80110215/Corrupted%20Raw%20Files/_X2_0503.CR2
https://dl.dropboxusercontent.com/u/80110215/Corrupted%20Raw%20Files/_X2_9844.CR2

 

[AshtonNekolah]

The followings are the links for good raw files that I copied from CFast to Mac's desktop:
https://dl.dropboxusercontent.com/u/80110215/Good%20Raw%20Files/_X2_0502.CR2
https://dl.dropboxusercontent.com/u/80110215/Good%20Raw%20Files/_X2_0503.CR2
https://dl.dropboxusercontent.com/u/80110215/Good%20Raw%20Files/_X2_9844.CR2

The followings are the links for good raw files that turned corrupted in hrs or days in CFast (I kept all the original raw files in CFast) which were good previously in CFast:
https://dl.dropboxusercontent.com/u/80110215/Corrupted%20Raw%20Files/_X2_0502.CR2
https://dl.dropboxusercontent.com/u/80110215/Corrupted%20Raw%20Files/_X2_0503.CR2
https://dl.dropboxusercontent.com/u/80110215/Corrupted%20Raw%20Files/_X2_9844.CR2

All files showed fine on my end.