There are good posts in here so I will try to avoid rehashing it. Instead I'll try to simplify the categories and weigh their relative importance (to me):
(The first three points are described more fully and clearly by Michael Reichmann at Luminous Landscape
- Microcontrast "sharpness." Also known as resolving power: The ability to resolve small details. Almost all lenses transmit acceptable levels of detail for full scenes in smaller prints or for compressed-size web use, but some photographers have stricter demands on their lenses (astronomers, bird photographers, and anyone who finds themselves cropping much of the image away to get a small portion of the scene).
- Whole-scene contrast (also appears as "sharpness"). Also known as acutance - how "acute" the difference between regions at a border appears. This is important because you want vivid scenes to appear vivid, not gray.
- Boke (short language note: A lot of people - probably most - add an "h" to the end; having studied Japanese, it is my opinion this promotes drawing out the word, or stressing one of the morae, the syllables): What I sometimes call "defocus area quality." The quality of the blurred area not in focus. Typical descriptions of boke are "nervous," "creamy," "distracting," and "smooth." Like any other optical effect, any type of defocus area quality can be useful in some situations and not useful in others, though the general preference is for boke to be smooth, continuous, and not have a spherical pattern (the blur area of some lenses is a bit like looking into a well, or having the centers closer to you). One thing is certain, however: Boke is the most reliable tool available to photographers to add structure to an unstructured scene, as it merely sketches out, in big blobs of light, the parts of the scene you wish to isolate from the subject (you hear a lot about the "subject isolation" capability of large aperture lenses). Can't reposition your camera to line up with architectural elements, or the scene is completely chaotic and you have to emphasize a relationship or downplay the chaotic totality of the scene? Get a wide lens and open it wide. Additionally, shooting in very low light is improved with wide aperture lenses. Almost always, photographers prefer to have lenses that do not detract attention from the elements in focus, while retaining the correct color of background elements.
- Control over optical aberrations
. Some of these aberrations affect blur quality, so we are focusing on other effects. Some common problems are purple fringing (sometimes I see this blamed on recent DSLR sensors) and green fringes. Some fringes are relatively easy to correct; others are more difficult. In the same vein, the ability of a lens to render straight lines as straight, without any distortion, is also important, though its cause is different. Different types of glass, lens coatings, and lens designs minimize these effects - one of the most sought-after designs is the apochromatic lens, which is designed to ensure that different wavelengths of light focus at the same point at the image sensor (no simple task actually).
- An even less typical criteria usually considered in lens IQ, but vitally important, is the presence or hopeful absence of focus shift
. In some lenses, when the aperture is reduced, the area which is rendered in sharp focus changes. Since DSLRs and even mirrorless cameras generally meter and focus wide open, this presents a real problem if you focus on a subject with the lens aperture wide open for a bright viewfinder and quick AF, but it turns out that the area in focus has changed after the aperture has closed for the shot (and you don't get any feedback suggesting this until you review the shot).
A couple other points: The susceptibility of the lens to flare
is important, especially for wide lenses, and in that case the type and quality of the lens flare spots can be considered, just like with boke. There are a number of effects tracing their origins to the aperture inside the lens, as well - some render "sunstars" from relatively bright light sources when closed down, and the number of points changes depending on the aperture construction. The out-of-focus circles (boke quality) is degraded if the aperture is not round - many older lenses with straight aperture blades render polygonal blur circles. And DO (diffractive optics) lenses, which operate on the same Fresnel lens principle as a lighthouse's lens, also can have unsatisfying blur quality (like a bullseye) and haloing - but they don't have to.