IPMS GSB judging system.

Who am I to Judge? (Or: I know art when I see it)

Howdy…

Today, I’m going to tackle a subject that I’ve skated around for a few years.  This topic, more than any other in scale modeling, can be the most polarizing thing there is in the hobby —the topic is judging scale models.  I will try to remain objective and neutral on my observations…

Full disclosure:  I am an IPMS/USA and an AMPS member.  My last full effort for competition in IPMS was the 2000 Space Coast show.  My last model on an IPMS competition table was at the 2005 Atlanta Nationals.  Why?  One, I subscribe to David Sarnoff's (the guy behind RCA and NBC back in the day) theory that "competition brings out the best in products and the worst in people".  I have been witness to more bent feelings, hot tempers, and bad blood at IPMS contests that I can shake a stick at, all because someone didn't get a big shiny to take home to prove to the world that he or she was The God of Styrene that week.  Two, the actual construction of a model falls under the heading of “craft”, but the final finish certainly borders on being art.  How do you judge art as a winner or loser?  

For the record, I have a rather large box of plaques and medals that I've won at model shows through the years, from a 3rd place plaque from an Embry-Riddle model show in 1983 to a Best Aircraft and “Best Between the Wars” plaque, a special award, from the 2000 Space Coast show, so this isn't being colored by sour grapes.  I build my models for me, and if they happen to garner some ugly plaque buildup, so much the better.  I don't do this for adulation, because after the show is over I still have to go to work and pay the bills...

Without getting into the weeds with the various systems out in the world used to judge a model show, I will instead take a look at the two most common systems used here in the United States.  They are the IPMS/USA system—commonly referred to as a “1-2-3” system, where every category (entries permitting) has a 1^st, 2^nd, and 3^rd place winner.

The other system is Open Judging, the best known being the system that AMPS uses.  AMPS doesn’t so much judge a model as they do score it, and multiple awards are possible within any given category.

These are some observations that I've collected over the years--some of them are actually mine, but most are from talking with others.  The general claims and observations are in standard type, my comments are in italics.

The IPMS system works like this—your model is placed on the table, in the relevant category.  Once registration closes and all the models are on the table, a team of judges evaluates each category.  They’ll look for basics: alignment, mold flaws, construction flaws, finish, detailing, etc.  If a model exhibits major flaws, they’ll be cut out of the running.  All the while, the models are compared to each other as well as to a mental “standard” that each judge is supposed to know and grasp.

The models that make the cut are again evaluated in the same manner but to a more focused look.  Eventually, the team arrives at the top four or five models.  The final cut is taking those four or five models and determining a 1^st, 2^nd, and 3^rd place winner.  Remember, throughout the process the models are not only being evaluated to the nebulous “standard”, they are also being compared to each other.

So, the observations:

IPMS judging picks “Winners”! (and, by extension, “losers”.)

If you "win", you get a big shiny trophy!  (But if you don't, you get bupkus.  This especially applies to competition newbies--they haven't yet grasped the nuances of how the deal works.)  

“If you want to know why your model didn’t ‘win’”, they’ll tell you, "ask a judge.”. (This is a noble effort, but it usually doesn’t result in anything.  Asking a judge is usually futile, since they want to get out and go home, too--it also seldom works, especially if the judge or  judges you consult weren't involved with your model, because part of the IPMS judging scheme requires the comparison of your model to the others on the table.  So, even if the judge that worked your category was there and remembered the way the judging unfolded, they really can't say for sure why you "lost", since they don't have the other models there for comparison.)

IPMS awards the modeler, not the model.  (Despite claims to the contrary from the IPMS Hard-Liners, the IPMS 1-2-3 system awards the model, not the modeler.  Think about it--if it rewarded the modeler, their model's standing in the show wouldn't depend on what else was on the table with it.)  
Under this system, a model could win Best in Show one week and get shut out the following week at the contest a few hours down the road.  (I’ve seen this first-hand, more than once.  Any repeat-ability is purely coincidental.)

IPMS judges learn on the fly from people who aren't always the best teachers.  (Most are very good at what they do, but I've come across a few guys who call themselves “IPMS Senior Nationals Judges” who still don't grasp what it is that the Society is trying to accomplish--they see it is a zero-sum, win-lose "bloodsport", damn the “casual hobby” aspects of it.  In short, they're bullies, bent on choosing only the models THEY deem as an appropriate “winner”.  You don’t see this often at the Nationals level, but it is still hanging around in the Local/Regional areas.  These guys are the ones who are insistent on judging accuracy, so you’ll know how to spot them…)

What the IPMS 1-2-3 system has going for it is speed--you can evaluate and judge a room full of models in a few hours.  A good team of judges can take a category of 20 models and determine the winners in less than 30 minutes.  (It also appeals to most Americans' desire to be called a “winner”, where 2nd place is the “First loser”.)

Now, let’s take a look at Open Judging (sometimes erroneously referred to a Gold-Silver-Bronze system) as employed by AMPS:

AMPS uses an open system where your models are placed in front of a panel of three or four judges and the model evaluated to a written "standard", and are judged in a “stand-alone” situation rather than being compared to the other models it competes with. (That standard merely quantifies the basics--alignment, construction, detailing, and finish--the very same basics that IPMS judges are taught to evaluate.  And note that neither organization judges accuracy.)

AMPS has several skill levels--Junior, Basic, Intermediate, and Advanced.  It is a ladder system—as your skills improve, you can be promoted to the next highest skill level.  This usually takes place at the annual AMPS International Convention.  (AMPS is one of the few hobby groups that also has a Master level--The National Model Railroad Association is the other, they feature a path to Master Model Railroader.  Rather than just proclaim yourself a “Master Modeler” because you’ve built a gazillion models, won at the Nationals, and maybe written a few articles for a magazine, in order to become an AMPS Master you must win Judge's Best of Show at the AMPS International Convention.  And, in order to do that, you have to be an Advanced member who earns a Gold at that show, and then go on to win Best of Show.  ONE person gets elevated to AMPS Master per year.)  

The judges write comments on the score sheet, and you get that score sheet back at the end of the show. (Most AMPS modelers want the score sheet with the judges’ comments more that they want the medal--we have one guy in our AMPS Chapter who routinely earns a fistful of medals every year, and he gives them back to us every year.)  

AMPS certifies their judges through a standardized training system.  (Up until a few years ago, it was administered by their Chief Judge, who would travel from show to show to train new judges, ensuring uniformity.  These days, there is a team of trainers.) 

The AMPS OJT consists of sitting for two shifts after you've received the classroom training.  Only then do you get your credentials.  (IPMS' OJT criteria aren't as stringent.  Not to say the way IPMS does it is "wrong" and AMPS is "right", just throwing it out there to compare and contrast.  IPMS judges only receive “credentials” after 20 years as a judge at the National Convention.)
In theory (and in practice 99% of the time), a model that earns a Gold medal in a given skill level at one show will earn the same medal at another AMPS show.  (In other words, there is documented repeat-ability in the system.)

Now, here are the drawbacks.  Open Judging, done correctly, takes time.  A lot of time.  We (AMPS Central South Carolina) host a show every year.  Registration opens at 8:30, the show runs until 5.  Judging starts at 8:30 and runs until approximately 3PM--and this is for around 100 models.  (Most IPMS local and regional shows draw a few hundred, and a National Convention draws a few thousand models.  Most AMPS local shows--they call them regional--draws around 100, and their International show draws a couple hundred for scoring.)

The associated drawback (for some) is how AMPS actually confers awards.  If you bring seven armor models, all in different categories, you have the potential to take home seven medals.  But if you bring seven Sherman tanks, all in U.S. Army markings, you will take home the highest medal awarded to your body of work because their all entered into the same category.  In other words, AMPS rewards the modeler and his or her efforts, not individual models.  So, AMPS is sometimes seen as limiting the number of models entered by doing this.  (Those who poo-poo that idea don't realize that AMPS has always had Display Only space at their shows, something IPMS has struggled with through the years.) 

The only place where someone at an AMPS contest can be a "winner" (in the "I'm Number One!" sense) is in the Best Of's.  The Best Of's are chosen by taking all of the Advanced level Gold medal winners in a given category and judging them in an identical fashion to the IPMS 1-2-3 system: count the flaws and compare between the eligible models.  Best of Show takes all the category Best Of's and does the same thing.  Otherwise, you are submitting a 3D research paper and getting a grade.

All that being said, is one system better than the other?  I don’t know.  It all depends on what is expected from a model show. 

If you want to be The God of Styrene for that week and have all the bragging rights associated with the title, if you want to be the Big Winner, you probably prefer the 1-2-3 system.  It is a system, as we have discussed, that picks winners and losers.  You won’t learn much from the exercise.

If you want to measure your abilities as a modeler, learn from your mistakes, and climb the ladder as you gain experience, the Open Judging system is probably the one you will choose.  You get that feedback from the judges telling you what they saw on your model, both good and bad.  You take that feedback and apply what you’ve learned to your next model.

Something else you ought to know--these days there are several IPMS Chapters who have used what they call a hybrid system, where the categories are evaluated as normal in a 1-2-3 system, then the top 5 or 6 models get the Open Judging score sheet treatment.  I don't like this--the people at the top really don't need the score sheet and comments, as they usually have a pretty good grasp on what they're doing (yeah, sometimes we "experienced" modelers make boneheaded mistakes and don't realize it, but usually we're on top of things).  The folks that don't make the cut are the ones who could really use the coaching.  Plus, by not treating all the models entered in the show the same way, you insert a double standard into the mix. 

I’ve seen some rather pointed comments made after a show here in the Southeast concluded, the group having adopted one of these hybrid systems.  The main comment from one person I know was (and I’m paraphrasing here), “I got two Silvers, but if I wanted critique on my models with feedback, I’d take it to the club meeting.  I want to know where I stand when I’m stacked up against other modelers.”

This all leads me to my last suggestion:  Before you enter any contest, read the rules applicable to said contest.  Don't like the rules?  Don't play the game.  It is as easy as that.  If you see the show is running an Open system, and you don’t like it, don’t play.  I can assure you that the show organizers aren’t going to change their judging system for you.

Finally, here’s some good reading material for you…

IPMS has their rules and a Modelers Guide to IPMS Contests available on their website.  They are downloadable, and are worth perusing.

Here’s a link to the AMPS system and philosophy.