My main interest was in answering some other questions.  The thought of duplicating David Kaleko’s study as I heard of it on Bill Butler’s LAT Crossword blog proved interesting.

As he did, I located time data on Bill’s NYT blog. I scraped through Bill’s blog from 03-01-2011 to 06-27-2017. I could have gone earlier than 03-2011, but it became hard to locate posts where he records times, as his first post to that blog illustrates. As well, this provides a greater number of data points than was available to Mr. Kaleko.

Given the tools I possess on my computer that I am fluent with (Delphi & spreadsheet), along with the other questions I wished to answer, I produced a CSV file to process against which could be manipulated to answer most of the questions I (or others) might have. These can be things like suggestions of grids to play in the archive if one has a NYT subscription, hardest grids, easiest grids, time periods for hard/easy publishing (I’ve wondered if it varies around certain events like the ACPT, since the NYT Crossword Editor, Will Shortz, manages this event), and the like.  If there is any demand for the CSV files, please let me know in the comments.

Sort Code, NYT Code, Day Published, Bill’s Time, Bill’s Time in Seconds
17-0624,0624-17,Saturday,Did Not Finish,-1
17-0625,0625-17,Sunday,23m 25s,1405
17-0626,0626-17,Monday,4m 56s,296
17-0627,0627-17,Tuesday,5m 31s,331

The second field is the NYT key code which they attach to all their puzzles. I reordered that so it could be sorted chronologically in the first field. The third is the day the puzzle appeared, the fourth is Bill’s specification of the time as it appears on the page.

After manually cleaning up the time data, I processed this file to add the time field. -1 specifies a puzzle which Bill either did not finish or did not record a time.

I split the data into separate files based on day of the week. Then I sorted each file based on time to aid more manual investigations.

After doing that, I sought to duplicate David Kaleko’s methods. I obtained the mean and standard deviation for each day within the spread sheet. Then I stripped the outliers (2*StdDev away from Mean) from each file. Stats below for that:

Monday: 1.52% records removed as outliers.
Tuesday: 3.95% records removed as outliers.
Wednesday: 4.27% records removed as outliers.
Thursday: 5.18% records removed as outliers.
Friday: 7.58% records removed as outliers.
Saturday: 8.79% records removed as outliers.
Sunday: 5.49% records removed as outliers.

In terms of the breadth of times, the more interesting part to note is how there can be a general variance of times from puzzle to puzzle. The New York Time puzzles are generally consistent, and Bill illustrates that in the Monday to Wednesday section. However the difficulty increases dramatically from Thursday to Sunday at times (but not others), increasing causing Bill’s times to vary, and producing a high standard deviation even when outliers are removed.

For instance, Bill fastest Friday solve is 8m27s, while his slowest is 68m06s. Thursday seems to be highly variable in difficulty, as it’s often known as “Tricky Thursday”, where changes in the typical crossword format are often published such as rebuses (multiple letters in one square) or other things where some kind of “trick” must usually be found out in order to complete the puzzle.  Usually in this puzzle, the time is often determined by how long it takes to discover this trick.  Friday and Saturday are themeless challenge days. The Sunday is a 21×21 grid, as opposed to the 15×15 grids typically offered throughout the week, where the claim is that it is at Thursday difficulty. There are occasional “tricks” in the Sunday NYT grid, but typically is a straight grid.

The Result
The next step is to duplicate the chart that Kaleko presents at the end. I do that by recalculating the Mean and standard deviation and then plot the times, plus or minus one standard deviation. Raw data appears below:

Monday 304.09 352.26 400.43 (5m 52s)
Tuesday 394.19 494.47 594.76 (8m 14s)
Wednesday 473.64 637.48 801.32 (10m 37s)
Thursday 723.31 1102.24 1481.17 (18m 22s)
Friday 822.46 1329.27 1836.07 (22m 09s)
Saturday 1176.17 1868.95 2561.74 (31m 08s)
Sunday 1266.15 1671.14 2076.12 (27m 51s)

So the traditional progression of difficulty seems to hold, though it is notable that Saturday on average took longer for Bill to solve than Sunday did, even though Sunday features a larger grid. I produced a chart which appears below, reflecting the New York Times solving times. As well, I reproduce the chart that David Kaleko produced for his LAT study.

How Hard Chart

While I don’t have data points of Kaleko’s LAT chart to plot onto this one to visually (or otherwise) compare, additional observations are interesting in comparing the relative difficulty of both puzzles. It seems the Monday NYT and LAT are roughly equivalent. The Tuesday NYT seems roughly equivalent to the Wednesday LAT. The Wednesday NYT seems equivalent to the Thursday LAT. Then the Thursday NYT seems equivalent to the Saturday LAT. Difficulty speaking. Personally, this is what I found when I started doing NYT grids regularly. In comparison to the LAT, the NYT grid seems to win out in its claims to be “the challenge” of crosswords.

Hopefully this was enjoyed, and the possibility to make some comparative observations would prove interesting.

3 thoughts on “An Analysis of New York Times Solving Times

  1. Fascinating article! I’m new to crosswords and have really enjoyed your blog. Would you be willing to share the csv file you reference above? It would be super helpful!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.