Why you don't ask about city populations

Oops

Back on October 16, the fourth and final round at Moose's Down Under consisted of a single question which provided players a list of 10 of the world's great megacities and asked each team to rank them in descending order of population.

I suspect that the question writer believed that the question, which could score anything from 0 to 10 points, would result in a point distribution resembling a normal bell curve. What it produced instead was a mushroom cloud. The approximately 13 teams playing combined for a collective two points.

Fortunately, even an accidental nuclear blast can be a teaching moment, and there are two lessons to be learned in this case: (1) don't ask questions about relative city populations; and (2) don't ask players to compare numbers that are really close to each other.

There are a host of reasons why, which I'll explain below, and demonstrate how Moose's question managed to trip just about all of them, a fact which virtually guaranteed disaster.

1. There are multiple ways to measure the population of a city, and none of them are authoritative or universally accepted

The predominant issue with relative city population ranking is one of definitions. When one talks about the population of a city, does one mean the population that lives only within the municipal city limits? Or should one ignore the imaginary line on Boundary Road and instead regard the city as an organic phenomenon, consisting of anyone who lives within a single socioeconomic aura? An even broader view defines a city as any blob or smear of contiguous urban area, even if it includes separate, non-overlapping auras.

Consider an everyday example that illustrates the problem: if your apartment is in Coquitlam, would you when abroad introduce yourself as a Vancouverite?

Unfortunately, which metric you choose makes a huge difference. According to the metropolitan agglomeration definition, Vancouver boasts over 3 million inhabitants and rounds out the great triumvirate of Canada's big three cities with Toronto and Montreal. If, however, you insist on formal city boundaries, our hometown scarcely clears 600,000 and finishes a mortifying eighth behind an ignoble crop of welterweight contenders including Calgary, Ottawa, and even Winnipeg.

This is by no means an unusual situation. Take a look at this handy table and notice that Tokyo may be the world's biggest city, or, perhaps, only 14th; Istanbul likewise is about 20th by metropolitan standards, but vaults to 2nd biggest by municipal boundaries.

Wolfmother

Hence much of the difficulty in Moose's puzzle. Tokyo and Istanbul were both among the cities teams were supposed to consider. Though players had no way to know this at the time, the question writer was operating from the municipal definition, which put the Japanese capital way down near the bottom of the list and old Constantinople near the top.

The lesson here, question writers, is that definition matters. You might attempt to get around this by specifying the metric in play, but this runs the risk of a) confusing players and b) coming off as excessively arbitrary (that is, if a player asks you why only this-or-that definition, what else can you say but "that's the one I chose so the question works," which is not much of an answer).

Therefore I think the better move is to use cities that are so far apart in population that their relative size remains the same in any system or combination of systems. As an easy example, Victoria-Boston-Sao Paulo leaves no room for error. Crafting a less obvious list will be a challenge for the writer, but can probably be done.

2. The margins for error were stupefyingly small

A quantitative look at the cities Moose's included in their question reveals that the differences between the population figures were so razor-thin that even if everyone had known exactly which population measurement was in play, there would not have been much improvement in scores.

I crunched some numbers using this list of municipal populations and discovered that, among the ten cities Moose's players were asked to rank, the average difference between each city and the next largest was only 7 percent, and only exceeded a 10 percent margin in two instances. Mumbai and Moscow, which ranked 4 and 5 on the list, have a difference of only 4 percent; Moscow and Beijing, 5 and 6, are only 2 percent different. The difference between 8th and 9th place was smallest of all: only 1 percent--less than 100,000 people in a total of 9 million--separates Tokyo from Mexico City. This is a small relative difference indeed, and seems even less significant when one considers the error factor built into the measurement, as well as the fact that city populations are changing all the time and at different rates.


Probably no one will disagree that distinguishing two figures with only a four-parts-in-one-hundred difference between them is, in general, too much to ask of players. I'd instead suggest 10 percent as an absolute minimum difference between figures if players are expected to compare them. This is based partly on my intuition and my experience in trivia, but the main reason being (and my apologies to the non-mathematically inclined) that my 10-percent rule means that players need never consider more than two significant figures.

In conclusion, the which-city-is-bigger-than-which question is a risky venture. Players generally don't know which population definition is in effect, which can introduce a kind of unintentional and artificial difficulty to the question.

Furthermore when players are expected to compare two or more numbers, those numbers can be close, but in my view should be different by no less than 10 percent. 

Moose's quiz is a superb one, but their cities question was a definite misfire. Still, it might have been fixed. In the first place, instead of selecting the 10 cities from the top 14, use the top 30 instead. This would both space out the margins between them and lessen the impact of using the wrong definition of city size.

Darby's now running monthly league competition

Darby's pub has introduced a new competition that adds an extra level of excitement to their already excellent product.

Throughout November, the results of each weekly quiz will also count towards a monthly leader board; total points scored carry over from week to week, and the team who tally the most at the end of the month win a spectacular prize pack including Steamworks beer and branded glassware, a Darby's gift certificate, custom printed winners' t-shirts with the team name screened on it, and a grand prize yet to be determined. (The prize has been awarded once already, and it was Whitecaps tickets, so expect something equally big).

Quizmaster Katie plans to repeat the Monthly Cup at least through the winter, and perhaps on indefinitely thereafter. As if you needed another reason to go.

This columnist's only complaint is that I keep declaring this or that pub to have the best prizes in the city, and then another goes one better and makes a liar of me. Truly are we living in a golden age of Vancouver bar trivia!

Darby’s runs every Thursday at the corner of 4th Avenue and Macdonald.

Cascade's What the #!*@? Quiz -- a mainstay of the Vancouver quizzing scene -- coming to an end

Sad tidings indeed from Main Street. Karen, the longtime quizmeister of the Cascade Room, has announced that effective next month, that establishment's venerable quiz will be no more--at least as a regularly-scheduled event. From their official facebook page:

Now for those of you who were not with us last night, an announcement! After more than 5 1/2 years, 165 quizzes and close to $30,000 raised for more than 70 charities, our What the #!*@? Quiz night is coming to an end. Our last quiz will be Monday, November 25th…please come and join us for a fond farewell…I've enjoyed almost every single second of it

I will continue with some special event Quiz Nights as well as Quiz Nights on request so keep in touch! This is not "goodbye"…it's "most likely see you soon". So I'll expect to see lots of familiar faces on the 25th xo k

For those who don't know, Cascade is one of Vancouver's longest-running and greatest pub quizzes. Many Vancouverites first caught the pub trivia bug there, including, I am sure, at least a few individuals who now run quizzes of their own. Yet none of them have ever duplicated the sheer quizzing pleasure of Cascade, which was probably best encapsulated in their famous "what's-in-the-shot" rounds, the likes of which we may never see again.

The continued success of Cascade owes everything to Karen's extraordinary commitment to her craft. Each week (and later, fortnightly, and eventually monthly) she would hammer out a full set of questions, the freshness of which was attested by the habitual inquisitions on the news of the day and other happenings of only a few days' vintage. Her annual Quiz of Champions, an invitational grand prix featuring only winning teams of the past year was yet another (hugely successful) expression of the same--for participants it was nothing less than the Wimbledon of Vancouver quizzery.

True to the progressive ethos of Mount Pleasant, 100 percent of proceeds from the quiz went to a local charity of the players' choosing, a fact which makes abundantly clear that the undertaking was, at bottom, a labour of love and not a mercenary ploy to pump up Tuesday night patronage (though that it certainly did).

After more than a half-decade of unflagging quality, the departure of What the #!*@? is to be sorely lamented. As mentioned above, Cascade's final regular quiz takes place November 25 and is sure to be a night to remember.