Todd MacMillan
Administrator

The intent of this thread is for former Digital Image Cafe Guest Judges to post their thoughts and insights as to the judging process. The judges from the past two years have been sent a request by me to post their honest opinions here.

Here is the note:
Dear Digital Image Cafe Guest Judges,
Cathy and I hope that you are well. I have a request. I would like to start a forum thread that contains your honest opinion of the judging system at Digital Image Cafe. All of you on this (blind) list have judged in the time that Cathy & I took on the site. October 2007 on.

It is my opinion that the information that you post will be of a great benefit to the membership that do not know the experience. I have received many an email that stated that you had enjoyed the process of judging.

To be perfectly honest, I want you to be honest about how it worked for you. What you learned, etc. Of course, comments about improvement are welcome as well. However, any improvements to Digital Image Cafe could not be scheduled for many months as we are currently booked with other projects. Quick fix issue, most likely can be done.

Obviously, you would be posting openly under your own account. There is NO obligation to comply with this request.

Here is the forum thread to use:
LINK HERE

Please take a moment to share your thoughts. The membership, Cathy and myself would love the insight.

Thank you again for the time which you had committed to The Cafe through judging!

Respectfully,
Todd

Ross Throndson
Cebu, Philippines

I judged last year...for me it was a great experience ! It's interesting just to see all the excellent images entered everyday. I choose one image that the photographer said had been entered for 3 yrs or so. Judging is very subjective, so if you have a photo you really believe in....Keep Entering It...I've experienced that myself. For me "Guest Judging" is what makes this site great ! You never know what might be chosen...it's not always just the same little group of Judges.

As far as a really bad picture being chosen there are a couple of safeguards. As a judge you can see what the other judges have picked. You can voice your disapproval of a particular photo to the Head Judge if need be. Also the Head Judge can tell you to select another one. That happened to me once.

If you get an opportunity to Judge..grab it..it's fun ! It gives you some insight into the "Photo Contest World"...It's just interesting to see it from the "Other Side."

Todd MacMillan
Administrator

Anonymous Judge:

The Cafe judging process would be substantially more effective if you made a few procedural changes. First, and foremost, the judging team should act as a team. The concept of allowing one judge to unilaterally push through an image to the front page, overriding the valid objections of other judges, is a big problem. By valid objections, I mean "seeing" problems with images containing technical and creative flaws embarrassing to the Cafe, if published. There should be a consensus of opinion by the judging team as to the worthiness of an image designated as POTD. It should be the responsibility of the entire judging team to "sign-off" on the entire slate of POTDs selected for the next day.

While anonymity of images in the voting process is pretty good, I observed several instances of images with identifiable styles chosen over images with far superior quality in a category. I could only assume a judge was selecting the image of a friend. Sorry for this harsh allegation, but it was the only conclusion I could draw. Associated to this dilemma is the posting of images in the critique corner, which are viewed by many, and subsequently entered into the contest. Judges should recuse themselves from choosing images when they know for certain the image's author. Image critiques are a good idea, but site publication of an image in the CC makes it "not anonymous" for the contest. Maybe, to earn the honor to be a judge, one would allow the CC to be inaccessible for 30 days prior to serving.

Oh, there are days when it would be nice (however unpractical) if an unedited version of an image could be viewed. While I certainly do not think this is feasible on a daily basis, it would be a great idea for images winning monthly or quarterly awards (with monetary value). It's an integrity issue. I think you may underestimate the lengths to which people will go to win a prize. And, I think you may overestimate the ability of a novice judge to spot the use of borderline procedures. I've seen quite a few landscape image POTDs with elements inserted which did not appear in the natural scene. I don't mind the techniques used, but these fantasy things belong in the Digital Art category.

The judging "window" is good. Sometimes, I felt the fairly large thumbnails "encouraged" some judges (in a hurry) to make their selection by only looking at the thumbnails. Yes, I know some people do not have high speed Internet, but if that constraint forces them to abdicate their responsibility of studying each and every image in full size, then those judges are not qualified to be judges. In general, judges should be very experienced, as measured by POTDs won. It's okay to have a minority of judges on the panel, but then the consensus of opinion by the judging team on selections becomes more important.

Thanks for asking my opinion. You have a good contest site. I encourage you to take chances to make it even better.

Todd MacMillan
Administrator

Anonymous Judge:

First of all let me say that Judging here is not only rewarding but very difficult!!! There are an amazing number of Front Page images on any given day in every category! Its Mind Boggling!
When I was a Judge I contested the choice of other judges and the Head Judge at the time told the judge who chose it and left it up to that judge to decide if it should be changed or not. I experienced that on a number of occasions, one time the judge even posted in the response, why, despite the flaws in the editing process that this image still won! I would love to see that if there is a question about an image that all of the judges are given the image to review along with precisely why it is being questioned and then a consensus of all of the judges would comprise the decision. I have also had an image questioned by another judge that I absolutely thought should stay so this would assure that it was not just 1 judge steering the pool. The site is evolving and attracting more members each day, I am grateful that this dialogue is even happening!
Thanks to Todd and Cathy for all that they do, this is a great site that should only get better!

Kay Beausoleil
Canada

What I remember most about judging was the joy of opening up so many excellent images every day and often, the frustration of being able to award only one POTD for the category. To compensate a little, I made it a point to use the thumbs available. (It's unfortunate the Theme category doesn't allow the thumbs.)

The judge's comments box was most useful to me, as sometimes I had trouble picking the winner from two or three images. I'd make notes as drafts of comments, and putting reasons on paper helped me make the final choice. Since I appreciate receiving a note from the judge on my images, I tried to explain myself for each POTD I awarded.

The judging procedure made perfect sense once given access to the judging page, and I found it easy to follow. The written instructions sent out ahead of time were complete but intimidating to this newbie judge because they were so detailed. I'd suggest that when you notify the future judge you simplify matters by saying that clear and full instructions will be posted on the judge's page, and leave it at that.

I found that communication with the Head Judge was surprisingly easy, given that even the HJ has a life beyond this site. Since all judges can have a say in others' picks, I did voice motivated objections a few times, and my opinions were fairly treated.

But I don't think it'd be practical to require all judges to sign off on all picks for the front page. This would mean that all eight would need to be available not only to do their own judging but also to do this second assessment after the other seven send in their selections, and at whatever time in the day this occurs. The Cafe doesn't need a full bench of the Supreme Court to set up the front page. The day's judge, the HJ and the contribution of whatever judges wander by should be sufficient. Otherwise, I see a coordination nightmare.

Having been a judge and been judged photographically in other places both real and virtual, I'm fully aware of how subjective judging is. What grabs me might be of no interest to you. What's artistic to me may be banal to you. All I can hope is that judges know a reasonable amount about photography and that they're honest and unprejudiced as possible in their assessments of the creative aspects.

However I do believe a standard of technical excellence should be maintained -- an objective evaluation to weed out the sloppy and the unfinished. IMO it's not sufficient for an artist to have something to communicate; the expression should enhance, or at least not detract from, the message. Judges need to consider not the photographer but the image itself as a creative and technical whole. I don't believe it's their job to encourage second rate photography for whatever reason.

In the categories, my one problem was DA. Since selective manipulation, according to the site rules, belongs in DA, I found it difficult to compare obviously artistic, creative work with images which looked like straight photographic representations.

This is a can of worms, I know, but it might be time to reconsider how much manipulation is acceptable in the regular categories in order to free up DA for real artistic expression. And particularly since the other categories haven't always been so pure: Topaz filter stuff was/is specifically allowed across the board (and Lensbabied images, too IIRC). I feel DA should be a place for creativity, not a place for technical skill in recreating the real. And please don't interpret this as a rant against Manipulation; it's certainly not. The only place it makes me queasy is in photojournalism. If the choice were mine, if it looks real it belongs in one of the other seven categories. Obviously, that view isn't unanimous, and isn't that of the owners of the Cafe. Different strokes for different folks.

Todd MacMillan
Administrator

Anonymous Judge:

Being a judge was a great learning experience for me. It can be quite overwhelming to choose just one for POTD. There are so many great entries each day. I like that we can give a "thumbs up" to encourage the other great entries. I think each judge has their favorite type of photography, so it is good we have to choose in a new category each day. outside our comfort zone.

Viveca Venegas
Puerto Rico

I love judging, but: I like better the idea of Judges voting in all or at least three categories. The photo would be better if chosen by points. Digital Photo Contest use to do it that way. For some reason many times the winners page have very good photos, others not so good, or simply candid shots. I prefer quality. I also think that photographers that have won more than 200 POTD should be in another trayectory, or maybe the MacMillans will have a Photo contest for Pro Photographers of multiple wins. Maybe the rules for those who fall over 200 wins [myself being close to it] be harder to achive. For example great photos straight from the camera, no manipulation, the type that is being awarded in Natures Best Magazine or National Geographic competitions. Manipulation is so carefully and well done, that most photos could in my opinion fall in DA category.

Robert Froton
United States

I judged here way back, and enjoyed it very much. The volume and quality of photos, and you can use that as a tool to help your own photography. It took quite a bit of time, I don't think I could do it with my work load now. I think most judges (I know I did) will open up most photos full size, but not all. Certain photos will grab certain people, we are all attracted to different things, but for the most part I believe all are gone through satisfactorily.

I picked one photo that was pulled, felt stupid for missing it but glad it was caught. Technical aspect is important, I know I sometimes even miss things in my own work (sometimes I think I am more subjective on other peoples work than my own).

I like the way the judging is set up here, with one judge making the decision and supervised by the head judge and fellow judges. I would not want a "pool" of judges deciding on one image, that would leave out the most important equation in judging, how it impacted that judge and made you feel, the personal devotion would be gone and would be more robotic. I would see what the other judges picked and sometimes would think "you've got to be kidding", seeing all the other possibilities, but you know, what I like, you may not, just like eating, our palates may vary greatly. All in all I think this is the best system for diversity, having a "group" decide will lose a lot of the emotional impact. Maybe we can all shoot the same scene and see who's is best?

Linda Yee
United States

I really enjoy and support the judging process here at DIC. I have had the pleasure of being a judge twice. It is a great learning experience and one that is well worth the work. It can be an almost overwhelming task to pick one image as POTD as being the best from so many deserving entries. As others have mentioned, I have had a couple of my choices questioned before they made it to the front page. In both cases, the concern was valid and I made another selection. I feel that this speaks to the quality control that is a part of the judging process.

We all occasionally miss something, but overall, I think the percentage of flawed images that make it to the Front Page is very, very small. In my case nothing makes an image more likely to make it to the front page than seeing an image that speaks to me artistically or emotionally. However, nothing makes me quickly dismiss that image as unsuitable for POTD as a preventable flaw like dust spots, crooked horizons or unfinished editing or a lack of technical quality like blocked up darks, hot blocked lights or lack of proper focus.

Additionally, I always felt comfortable in questioning a selection or two where I saw the need. In one case, I had a question about whether a plain white background on a macro was natural or not because it had no shadow. However, Todd explained some of the ways that this could be accomplished without resorting to painting in a background and let the image stand. I tried one of the processes that Todd suggested (using a light box as a background) and loved the result. Since then, I have had a couple of POTDs using that technique. :) In all cases, my concerns were addressed with care and fairness by Todd (or Ron when he was Head Judge).

I do support the current system of judging where each judge makes his own selections from his category for the day. I think that this feature keeps the picks "fresh" and varied from day to day and month to month.

Robert Ganz
Canada

I have judged here on a number of occasions and have found it to be a rewarding learning experience. I have defended my choices when needed, and have had my choices pulled, and to me it is simply a continuation of the photography learning experience that I embarked on the moment I picked up my first camera and said to myself "I want to do this!".
The exposure to the pool of talent that submits here at the Cafe given to you when you judge is horizon-expanding. The process of picking what you think is the BEST photo in that category on that day is a challenge that calls to the table all you have learned and begs you to apply it and further to go beyond what you know technically and enter the zone of trying to look at the world through another photographer's eye (get out of the comfort zone behind your OWN lens). Having said all that (which may be preachy to some!) there seems to be a constant discussion here about conformance to standards and technical quality vs art (DA, or otherwise).
I don't want to discuss whether or not manipulation should be allowed in all categories (officially) or not. The site is currently very clear on what the rules are, but not all judges are adept at spotting the infractions....just because I have a certain number of wins in NO WAY qualifies me to say for sure if a photo was manipulated outside of regulation or not (thank goodness for some peer review, and a head judge who keeps things on an even keel!)

I did want to talk a bit about the subjectiveness inherent in my judging process. It cannot be avoided, and if this admission eliminates me from future judging invitations, well so be it. :)

I was always a much harder judge when evaluating shots for which I knew there was ample time to get it right. Studio shots, set piece shots...IMHO there is just no excuse for poor composition, lighting errors or focus issues (I am pretty unforgiving). Even for action, landscape or macro shots there is no excuse for not doing the post processing that *is* allowed.....horizons should be level, dust spots should be addressed, and the chosen POV and DOF had to be appropriate and *work* within the context of the photo. On the other hand I took into account the difficulty or challenge in getting the shot as well (this view has been contested by those who think that a shot should be technically perfect above all else)....I could not help it.

I like the judging process here at the Cafe. Honestly I prefer it to a system that relies on a general membership that votes....all too often they turn into popularity contests. I think that the rotating pool of judges overseen by a head judge is about as fair as it can get, particularly now that judges can look outside their own categories and question other's choices (if needed).

If I could suggest anything to change, it would be to perhaps take into consideration ranking more than a first choice, and then let a smaller group of judges choose winners from the reduced pool of photos. Would this mean that we would need a two-day lag between choices being made, and their posting on the site instead of the one day turn around as it is now?
For sure it would help on those days when as a judge you are faced with literally dozens of potential front-page shots and have to choose THE ONE.

For those that have never judged, I encourage you to try it. For those that have judged and continue to judge, thank you for your commitment and the daily effort you put into the process.

Cheers,

Robert