Suggestions for testing R6/R5 IBIS?

YuengLinger

Print the ones you love.
CR Pro
Dec 20, 2012
3,751
2,269
USA
I'd like to do some simple tests of IBIS, but if I do them on a tripod, would the results be meaningful? And if I did them handheld, how do I convince myself, much less Canon CPS, that my technique is not causing problems?

I'm beginning to see pretty consistent problems with slight motion blur in the 1/250th-1/600th sec. shutter speed range when IBIS is on. It can happen on an EF lens with IBIS, such as the ef 100mm f/2.8L IS Macro which I use for walkaround and portraits sometimes, or on the EF 35mm f/1.4L II. It's happening in both ECFS and mechanical shutter modes, so I'm definitely suspecting IBIS--not shutter shock.

I've suspected some issues with RF lens IS for a while now, even when they are on the R, but I'm consistently seeing these issues with adapted EF lenses on the R6. I'll get one shot in focus, subject standing still, and then the next two or three just showing a little blur, not tack sharp, and then the next few sharp again. This is happening in bursts where I don't think I'm jerking while holding the camera, but how can I be sure?

My idea is to use a mannequin to take shots with IBIS on and off, but if I'm doing this handheld because that's when we use IBIS, how do I eliminate handshake as a significant factor? Just keep repeating until I have a clear trend (or not)?

Thanks!
 
  • Like
Reactions: 1 user

Joules

doom
CR Pro
Jul 16, 2017
1,801
2,247
Hamburg, Germany
My idea is to use a mannequin to take shots with IBIS on and off, but if I'm doing this handheld because that's when we use IBIS, how do I eliminate handshake as a significant factor?
Correct me if I'm wrong, but isn't the whole point of IS / IBIS to reduce the impact of jitter / shakiness in the hands on the image quality? If you remove that factor out of your testing, what exactly is it you are testing?

If there are shutter speeds that the IS / IBIS is less effective or even harming the results, I would expect that to show up in a comparison like this:

You take a number of shots per setting to be evaluated. Maybe 10 to 30 images per setting. The same amount for each. Then go through them and rate their quality.

s = sharp, image is as crisp as you would expect with the lens and aperture combination in use
m = mediocre, image is not really blurry but shows lesser contrast on edges compared to a sharp one
b = blurry, image does show some clear sign of blur, maybe even to the degree that the kind of blur (motion, focus, diffraction) can be determined

Format the results for better presentation in a way similar to the following table. Keep every numeric value in a seperate cell if you want to employ some software evaluation techniques. I combined them here for clarity of what belongs together.

Shutter speed% s / m / b without any stabilization% s / m / b with only IS% s / m / b with IS + IBIS
1/10025 / 50 / 2535 / 50 / 1550 / 50 / 0
1/50050 / 30 / 2060 / 30 / 1070 / 30 / 0
1/100070 / 20 / 1080 / 15 / 590 / 5 / 0

If you suspect IBIS to be the culprit, involving the R in the testing may help. But to keep variables like the differenc in resolution and low pass filter in use the same, using just the R5 may make things easier if there is a clear issue. Anyway, if the data is presented in a form as above, I would expect the proportion of sharp shots to alway increase if you move left to right through the table, or top to bottom. I'm not sure how the slope of the increase should look, but you could plot that based on the numbers as see if there is some weird plateu or kink when looking at the IBIS data.

Not sure how many shots you'd need per cell of the table, but in any case it highly depends on how much effort you are willing to spend and how precisely you want to nail down the issue. Using only 3 shutter speeds for example may not be particularly insightfull if issues appear only in a narrow range of settings.

What I also would note is to use a plane with some texture on it (Like stripes printed on carboard, or something along those lines) and angle that plane with regards to your shooting. So if blur is caused by problems with focus, that can be detected by looking at where the point of focus is along that plane (IBIS may indeed have an impact here as the sensor DOES move relative to the lens elements, after all).
 
Last edited:
  • Like
  • Love
Reactions: 3 users
Upvote 0

YuengLinger

Print the ones you love.
CR Pro
Dec 20, 2012
3,751
2,269
USA
Correct me if I'm wrong, but isn't the whole point of IS / IBIS to reduce the impact of jitter / shakiness in the hands on the image quality? If you remove that factor out of your testing, what exactly is it you are testing?...
Great! Thank you! I have a project this week. And I think, also, taking some "control" shots, with IBIS off, on a tripod at very close to the same perspective/distance as the handheld shots will give me a baseline.

The only way I know to do this with "only IS" is to use the R. I don't think we've had a firmware update giving us an option to turn one off without the other.

One other thing: I think I have a method that suggests whether a general softness is being caused (or worsened) by motion blur--rather than just AF problems. I know somebody will correct me if I'm wrong! But some of these shots that have been concerning me are improved amazingly by PS CC's Filters>>>Sharpen>>>Shake Reduction.

In the attached examples, cropped to approximately 200% to show what is happening with the pixels, if you can toggle between them you will see that all the pixels get shifted a tiny bit to one side. If it were just the boy moving, I'd expect the print on the bags to camera right would not be improved--they'd be shifted to a less sharp position. But they all get moved the same, and the whole image was sharpened.

I've noticed this before and now believe, when Shake Reduction works, which isn't always, it detects slight motion blur and shifts the pixels. If it were just AF being off, or general lens softness, would this be the case? Would the Shake Reduction be as effective?

Btw, image taken with the R6 + ef 35mm f/1.4L II, f/2.2, 1/400th sec, ISO 1250, EFCS.
 

Attachments

  • No sharpening.jpg
    No sharpening.jpg
    773.6 KB · Views: 195
  • With sharpening.jpg
    With sharpening.jpg
    990.1 KB · Views: 192
Upvote 0

AlanF

Desperately seeking birds
CR Pro
Aug 16, 2012
12,356
22,535
If you want to look for motion blur, then use a proper target with graduated vertical and horizontal lines, concentric circles, and stars, like this one from Bob Atkins. That way, you will readily detect motion or shutter slap by differential resolution in the horizontal and vertical directions. I test my lenses hand held at different shutter speeds to know which ranges I can use without motion or mechanical blurring.

BobAtkins.jpg
 
  • Like
  • Love
Reactions: 3 users
Upvote 0

Joules

doom
CR Pro
Jul 16, 2017
1,801
2,247
Hamburg, Germany
One other thing: I think I have a method that suggests whether a general softness is being caused (or worsened) by motion blur--rather than just AF problems. I know somebody will correct me if I'm wrong! But some of these shots that have been concerning me are improved amazingly by PS CC's Filters>>>Sharpen>>>Shake Reduction.
I don't know if you'll have much success with that. I just tried that filter and it definitively adds contrast to edges and thereby adds to the impression of sharpness even in pictures with 0 motion blur. I just took a picture of a chart I had at hand in silent mode (EFCS) with my 80D + Sigma 35 mm Art @ 2.8, 1/25 s ISO 100. From a tripod, with 2 second timer. The chart rested on an office chair, so I took one picture with the chair at rest and one with it gently spinning to introduce some simple motion blur.

Attached are gifs showing the results for those two shots, where I left one unprocessed (0 sharpening and noise in LR, no profil corrections. That applies to all of these), one processed by the filter you mentioned, with the default settings, and one tweaked manually with smart sharpen to deconvolve some details (no enhancements of contrast beyond recovering actual detail).

With the chair spinning:
blur.gif

Without any motion blur:
sharp.gif

You can see the Shake reduction filter applies thick halos along the edges, and moves the image, despite no motion blur being present in the lower images.

What Alan proposes sounds much more reliable and allows you to make judgements without using additional software. I think it will work best with the chart parallel to the sensor though, so to check for front / back focus and focus tilt, my proposed angled chart with lines also seems a decent supplement.
 
Last edited:
  • Like
Reactions: 1 users
Upvote 0

snappy604

CR Pro
Jan 25, 2017
681
641
whoa... glad someone is taking this on.. I now have my EF adapter and using 3rd party glass (specifically sigma 150-600 C ) and noticed exactly in that range I was getting way more unexpected blur from eagles but I hadn't had time to see if it was my technique or not.. I was shooting in poor conditions, but would've thought 1/640th of a second with me panning should've got more consistant results.

I also was thinking maybe 3rd party might not communicate something correctly so was going to test with IBIS off vs on.... really want to see your results.

For tests with tripod, shouldn't IBIS be off? I seem to recall I.S. is recommended to be off on tripod work, would expect same from IBIS? it's always micro adjusting from I understood and on a tripod its a lot more still.
 
  • Like
Reactions: 1 user
Upvote 0

AlanF

Desperately seeking birds
CR Pro
Aug 16, 2012
12,356
22,535
whoa... glad someone is taking this on.. I now have my EF adapter and using 3rd party glass (specifically sigma 150-600 C ) and noticed exactly in that range I was getting way more unexpected blur from eagles but I hadn't had time to see if it was my technique or not.. I was shooting in poor conditions, but would've thought 1/640th of a second with me panning should've got more consistant results.

I also was thinking maybe 3rd party might not communicate something correctly so was going to test with IBIS off vs on.... really want to see your results.

For tests with tripod, shouldn't IBIS be off? I seem to recall I.S. is recommended to be off on tripod work, would expect same from IBIS? it's always micro adjusting from I understood and on a tripod its a lot more still.
I found AF with the 150-600mm C on the R5 rather slow, even with upgrading the firmware. I never use shutter speeds as low as 1/640s for BIF.
 
  • Like
Reactions: 1 user
Upvote 0

snappy604

CR Pro
Jan 25, 2017
681
641
I found AF with the 150-600mm C on the R5 rather slow, even with upgrading the firmware. I never use shutter speeds as low as 1/640s for BIF.
fair ... there were enough variables in which it could've been me..but I've gotten results with my 80D and the same lens at those speeds that were good..

Some shots were in focus but a dismal rate. At a glance there was some further blur on either side of the body making me think I was in focus (read depth of field blur on either side of the body/eyes) but something else causing the blur. Again too many variables, but its not the first time I thought I saw some oddities. Was thinking of more structure similar to what is being described here, so pleased to see someone else do it as well.

like any tool, need to figure out what it can and can't do.. so I'm pushing a bit, but thought that was a safer range. 1/1000th of a second had better results. Eagles tend to be a bit lazy flying :)
 
  • Like
Reactions: 1 user
Upvote 0

YuengLinger

Print the ones you love.
CR Pro
Dec 20, 2012
3,751
2,269
USA
If you want to look for motion blur, then use a proper target with graduated vertical and horizontal lines, concentric circles, and stars, like this one from Bob Atkins. That way, you will readily detect motion or shutter slap by differential resolution in the horizontal and vertical directions. I test my lenses hand held at different shutter speeds to know which ranges I can use without motion or mechanical blurring...

Thank you, Alan. I'm printing five of these on 4x6 glossy now to glue to a foam board. I'm sure this won't be the last time I want to test lenses, IBIS, AF.
 
Upvote 0
I'm beginning to see pretty consistent problems with slight motion blur in the 1/250th-1/600th sec. shutter speed range when IBIS is on.

If you are seeing motion blur at 1/250, then it's difficult to believe that it is related to the shaking of the camera body, unless you're on a roller coaster. The original poster did not indicate what he/she was taking a picture of but I'm assuming that it was a still object. If the image is blurred then the only logical reason is a defect in the IBIS algorithms used to correct for body shake. I would try both on and off tripod shots of a resolution chart with and without IS turned on. I think the only way to turn IBIS off is by turning IS off.

I have seen previous posts about IBIS issues at higher shutter speeds but don't remember where. Personally, I have never noticed the problem with my R5 and any of my RF lenses 15-35 f2.8, 24-70 f2.8, or 100-500 but without testing, don't have a dog in this fight.

Have you checked the focus of the lens to be sure it is not front or back focusing on the subject? If the aperture was set at a wider open setting and the lens focused before/after the subject, you could be mistaking out of focus for motion blur due to DOF issues?
 
Upvote 0

YuengLinger

Print the ones you love.
CR Pro
Dec 20, 2012
3,751
2,269
USA
If you are seeing motion blur at 1/250, then it's difficult to believe that it is related to the shaking of the camera body, unless you're on a roller coaster. The original poster did not indicate what he/she was taking a picture of but I'm assuming that it was a still object. If the image is blurred then the only logical reason is a defect in the IBIS algorithms used to correct for body shake. I would try both on and off tripod shots of a resolution chart with and without IS turned on. I think the only way to turn IBIS off is by turning IS off.

I have seen previous posts about IBIS issues at higher shutter speeds but don't remember where. Personally, I have never noticed the problem with my R5 and any of my RF lenses 15-35 f2.8, 24-70 f2.8, or 100-500 but without testing, don't have a dog in this fight.

Have you checked the focus of the lens to be sure it is not front or back focusing on the subject? If the aperture was set at a wider open setting and the lens focused before/after the subject, you could be mistaking out of focus for motion blur due to DOF issues?
It's happening with different lenses, but seems to be showing more on EF lenses. I notice the issues most from 1/400th to 1/800th. But before talking to CPS, I'm going to conduct careful tests, helped by Joules's method suggestions, and AlanF's link to a very useful test target. The camera in question right now is my R6.

Unless what I do can be reproduced by techs at Canon, they won't see it as an issue. I ask myself why I'm getting soft shots at good shutter speeds, an issue I never saw on the R, but there are so many variables! My technique, shutter modes, IBIS, lens IS, AF... I'd like to nail this down!

I decided to go with a block of nine 4x6 prints for a test target, all permanently glued to a foam board. Seems like too many, right? But this way I've got all my edges covered for other types of tests in the future! Right now a few prints are drying, and then I'll glue them in place, and then, hopefully by the weekend have some results to share.
 
  • Like
Reactions: 1 user
Upvote 0

Bdbtoys

R5
CR Pro
Jul 16, 2020
467
331
YuengLinger,

Could you try asking Canon for the 1.0.0 firmware for testing?

I remember someone mentioning shutter shock after updating to 1.1.0... that it wasn't there before. 1.0.0>1.1.0 introduced some IBIS changes... so perhaps you could start there and compare firmware just to rule that out (would be easy to convince Canon to do something if you show before/after firmware pictures). It might save you a lot of time if it's as easy as the firmware.
 
Upvote 0

YuengLinger

Print the ones you love.
CR Pro
Dec 20, 2012
3,751
2,269
USA
YuengLinger,

Could you try asking Canon for the 1.0.0 firmware for testing?

I remember someone mentioning shutter shock after updating to 1.1.0... that it wasn't there before. 1.0.0>1.1.0 introduced some IBIS changes... so perhaps you could start there and compare firmware just to rule that out (would be easy to convince Canon to do something if you show before/after firmware pictures). It might save you a lot of time if it's as easy as the firmware.
Sorry, I've updated my firmware already. I will look at older images, but I've only had the camera about a month.

I can really only perform tests with the firmware currently installed.

I do not have any contacts at Canon. If a problem does exist, Canon will have to be convinced by many photographers who have taken the time to perform simple tests that can be reproduced, or provide other clear evidence.
 
  • Like
Reactions: 1 user
Upvote 0

Bdbtoys

R5
CR Pro
Jul 16, 2020
467
331
Sorry, I've updated my firmware already. I will look at older images, but I've only had the camera about a month.

I can really only perform tests with the firmware currently installed.

I do not have any contacts at Canon. If a problem does exist, Canon will have to be convinced by many photographers who have taken the time to perform simple tests that can be reproduced, or provide other clear evidence.

Tech support may supply the firmware if asked and you tell them what you're doing. Not an issue to flash back... do the test, and flash back to current.
 
Upvote 0

Joules

doom
CR Pro
Jul 16, 2017
1,801
2,247
Hamburg, Germany
Tech support may supply the firmware if asked and you tell them what you're doing. Not an issue to flash back... do the test, and flash back to current.
I think showing that there IS a problem in the current firmware is more important than showing that there may not be one on the earlier ones.

If he uses an earlier one, and finds no issue, the existence of a problem has not been proven and therefore Canon is very unlikely to put any time or effort into the matter while they have bigger issues on the backlog.

If there is an issue in the current firmware though, that can be demonstrated (and reproduced), I would expect Canon to do their job and spend the time and money for investigating where islt comes from themselves.
 
  • Like
Reactions: 1 user
Upvote 0
Mar 25, 2011
16,848
1,835
There is a test method, here is a draft copy. Obviously, The average person is not able to do this, but understanding how it is done might at least let someone devise their own method.

Image Stabilization Document CIPA

There is also a thread on DPR discussing correction capability away from the center.

Theoretical limits of IBIS away from the center of an image: Photographic Science and Technology Forum: Digital Photography Review (dpreview.com)

Finally, Jeffrey Fridel did a tripod stability experiment several years ago. I think his method might be adapted to test IBIS by turning ibis on and off. There are 3 parts. The jist of his testing was that cranking up the center column created a problem if you need good stability. Crank up your center column and compare? You might get some repeatability by dropping a 5 lb bag or box on the floor at a specific spot to get the same vibration each time (within limits).

Jeffrey Friedl's Blog » Tripod Stability Tests, Part I (regex.info)
 
  • Like
Reactions: 1 user
Upvote 0

YuengLinger

Print the ones you love.
CR Pro
Dec 20, 2012
3,751
2,269
USA
There is a test method, here is a draft copy. Obviously, The average person is not able to do this, but understanding how it is done might at least let someone devise their own method.

Image Stabilization Document CIPA

There is also a thread on DPR discussing correction capability away from the center.

Theoretical limits of IBIS away from the center of an image: Photographic Science and Technology Forum: Digital Photography Review (dpreview.com)

Finally, Jeffrey Fridel did a tripod stability experiment several years ago. I think his method might be adapted to test IBIS by turning ibis on and off. There are 3 parts. The jist of his testing was that cranking up the center column created a problem if you need good stability. Crank up your center column and compare? You might get some repeatability by dropping a 5 lb bag or box on the floor at a specific spot to get the same vibration each time (within limits).

Jeffrey Friedl's Blog » Tripod Stability Tests, Part I (regex.info)
Scanning all three, they look very interesting and informative. As I'm a portrait and event photographer, plus landscape for personal enjoyment, I'm not concerned at this time with the question of tripod use and IBIS/IS. (My habit is to turn off stabilization when the camera goes on a tripod, but I don't photograph enough wildlife to think it through much.)

If you or others see a significant problem with what I lay out below, please let me know!

I've just finished with one of three lenses (Rf 50mm, Rf 24-70mm, EF 35mm f/1.4 II), using a common target and the same exposure over a range of shutter speeds, from 1/60th to 1/2000th. So, as my shutter speed gets faster, my ISO goes up also. My histogram looks nearly identical for many, many shots.

In order to include/eliminate several factors, I tested Drive Modes ( single shot, and the three burst speeds) and Servo vs One-Shot AF. Plus I did one series on a tripod with IS Off and One-Shot AF, which, for my purposes right now is good enough for a control set. To save time I just took five shots in each Drive Mode. (With high-speed burst I just went by what sound like five-seven shots, it's so fast.)

I'm doing these tests to see for myself, under controlled conditions, with a stationary subject, with no pressure to hurry up and move on, etc., whether IBIS is causing softness from 1/400th and 1/800th, shutter speeds I've never had trouble or doubts with using non-IBIS bodies. Of course they aren't perfectly controlled! Handholding itself is so variable, even for one person. Some parts of the day I know my hands are shakier than others, for example, and I can't really take the whole array of shots in a single session. (I just never get that kind of block of time anymore!) But I'm being as consistent as I can be.

If the results show nothing conclusive, well I'd have to dig deeper--if during real world shooting I keep seeing issues. If my simple tests strongly suggest certain shutter speeds are unexpectedly soft with IBIS on, then what I will share here might encourage others to try their own tests, and, more immediately, be something I can send to CPS, as in the past they have been willing to reproduce shooting scenarios.

In any event, I've just tucked in the kids. I'm eager to look at the huge set from the first lens.

Thanks, all, for the suggestions and encouragement! I hope to have something useful posted by the weekend!
 
Upvote 0