2014 Crypto Challenge

In 2014, the Khan Academy had a cryptography challenge, called the 2014 Cryptochallenge. It consisted of a narrative leading to clues that were progressively more difficult to decode. The idea was to introduce people unfamiliar with ciphers and cryptography to some basic ciphering methods through examples and clues. I stumbled upon it well after it was debuted, but still had fun going through the challenge.

Please note that while I am displaying the code I wrote to decode the messages, and describe the methodology to arrive at the answers, I will not reveal the decoded messages because I don’t want to ruin the fun for others.

It all starts with the final clue, actually. In the challenge, the goal is to decipher this message that is found:

clue_4b

After finding out about a pair of burglars in the vicinity, you find a ciphered communique. The goal is to decode the message by finding clues left behind the pair of villains who communicate using various ciphers. Although this main message is the one we want to decode, we have to work through their previous communications in order to figure out the method of encoding.

Clue 1

clue_1

The first clue is a simple cipher. Khan Academy gives us the clue that it is a Caeser Cipher, which is where the value of each letter is shifted n places. For example, we could encode the word “CAT” by shifting all the letters 3 values, and end up with “FDW”. Although easy to decode if you know the method, to someone who has no idea, it will just look like gibberish.

This is simple to brute force, as there are only 25 ways the shift the alphabet. I wrote the following C code to decipher the message in all 25 ways, and then I looked through the results for reasonable output.

void cipher(const char *input, char *output, int shift) {
    for (int i=0 ; i<=strlen(input) ; i++)
    if ('A' <= input[i] && input[i] <= 'Z') //Capitals
        output[i]=(input[i]-'A'+shift)%26+'A';
    else if ('a' <= input[i] && input[i] <= 'z') //Lowers
        output[i]=(input[i]-'a'+shift)%26+'a';
    else
    output[i]=input[i];
}

int main(int argc, char *argv[]) {
    const char *messageA="vwduwljudeehghyhubwklqjlfrxogilqgsohdvhuhwxuqdqbeoxhsulqwviruydxowdqgdodupghvljqedvhgrqzklfkedqnbrxghflghrqldpvhwwlqjxsvdihkrxvhfr";

    const char *messageB="gluhflishjrvbadvyyplkaohavbyjpwolypzavvdlhrvuuleatlzzhnlzdpajoavcpnlulyljpwolyrlfdvykpzaolopkkluzftivsvmklhaoputfmhcvypalovsilpuluk";

    for (int i=1 ; i<=26; i++) {
        char outputA[strlen(messageA)];
        cipher(messageA, outputA, i);
        printf("1/2, Offset: %d\n%s\n\n", i, outputA);

        char outputB[strlen(messageB)];
        cipher(messageB, outputB, i);
        printf("2/2, Offset: %d\n%s\n\n", i, outputB);
    }

    return 0;
}

Clue 2

clue_2

For this message, they give us the clue of Polyalphabetic Cipher. For this type of cipher, there is a key word that is repeated. Each letter of the alphabet is assigned a value based on it’s order (A=0, B=1… Z=25). The value of the key letter is added to the value of the message letter. If the value is greater than 25, you wrap the value around back to 0 and continue. From Wikipedia, we see this example:

Plaintext:  ATTACKATDAWN
Key:        LEMONLEMONLE
Ciphertext: LXFOPVEFRNHR

A (the 0th letter of the alphabet) plus L (the 11th), when added together would produce 11, which is L.
T (19th) plus E (4th) equals 23, which corresponds to X. Etc.

To reverse the cipher, all you need to do is subtract the values of the keyword from the scrambled text.

The key we will use was alluded to in the decoded first clue. For this clue, the cipher key was doubled; that is each letter of the word was used twice before advancing to the next letter. This is apparently a common practice. What tipped me off is that the thieves previous communication began with START, and ended with END. I presumed that they might do the same with this message. By subtracting the supposed value START from the scrambled message, I could confirm that the keyword was in fact doubled, and then plugged it into the program I wrote to decode the whole message.

void cipher(const char *key, const char *input, char *output) {
    int n = 0; // Key index

    for (int i=0 ; i<strlen(input) ; i++) {
        int char_enc = input[i] - 'A'; // Get encoded char value
        int char_key = key[n++ % strlen(key)] - 'A'; // get key char value
        int char_dec = (26 + char_enc - char_key) % 26; // Reverse vigenere cipher
        output[i] = char_dec + 'A'; // Store deciphered message
    }
}

int main(int argc, char *argv[]) {
    const char *code="SSKKUULLLL"; // Doubled letters for key is common
    const char *message="KLKBNQLCYTFYSRYUCOCPHGBDIZZFCMJWKUCHZYESWFOGMMETWWOSSDCHRZYLDSBWNYDEDNZWNEFYDTHTDDBOJICEMLUCDYGICCZHOADRZCYLWADSXPILPIECSKOMOLTEJTKMQQYMEHPMMJXYOLWPEEWJCKZNPCCPSVSXAUYODHALMRIOCWPELWBCNIYFXMWJCEMCYRAZDQLSOMDBFLJWNBIJXPDDSYOEHXPCESWTOXWBLEECSAXCNUETZYWFN";

    char output[strlen(message)];
    cipher(code, message, output);
    printf("%s\n", output);
    return 0;
}

Clue 3

clue_3

Clue three was a tough one. I had to look up the hint, which points to a Kahn Academy video called “visual telegraph.” Within the video, we learn of a character encoding method called “Polybius Square.” It explains how characters can be communicated using pairs of numbers that describe a row and column in a character lookup table. Interestingly, in this clue, the message is composed entirely of numbers! We can assume that the table of characters is five by five in size because we never see any numbers smaller than one or larger than five. Additionally, we can also assume that the message starts with the word START again. This can help us figure out the orientation of the lookup table. I made a program to take pairs of numbers and print out the message using the table, and it worked like a charm!

void cipher(const char *input, char *output) {
    char key[5][5]={{'A','F','K','P','U'},
                    {'B','G','L','Q','V'},
                    {'C','H','M','R','W'},
                    {'D','I','N','S','X'},
                    {'E','J','O','T','Y'}};

    for (int i=0 ; i<strlen(input)-2 ; i=i+2) {
        //Get coordinates from key grid
        printf("%c",key[input[i]-'1'][input[i+1]-'1']);

        //Save ascii to output
        output[i]=key[input[i]-'1'][input[i+1]-'1'];
    }
}

int main() {
    const char *message="44541134541123335344541242434244325141212311311353155442544244424344325141534354324234411125513553341342432253431144543453432251343142143251341253341215541534513351444411225144425442444415345123551543213451111311212351425431533321424351445315341434512542531544335154325341443043513544";

    char output[strlen(message)];
    cipher(message, output);
    printf("%s\n", output);
    return 0;
}

The Final Clue

The final clue stumped me for a while. I ended up getting busy and forgetting about the cryptochallenge. A few months later I found the files on my computer and resumed where I had left off.

Thanks to the previous messages, we found a “safe house” that yielded clues to how this final cipher was put together. Some key insights:

  • The message is converted to digits using a method similar to clue 3
  • There is a binary key which is obtained from a newspaper clipping where consonants are 0 and vowels are 1
  • The symbols represent 2 digits; 1 digit for the vertical/horizontal and 1 digit for the diagonal positions
clue_4c
Clue indicating glyph’s numerical values
clue_4d
Clue indicating glyph’s numerical precedence
clue_4f
Clue indicating visual telegraph layout, and process for ciphering
clue_newspaper
Clue used for cipher pad

There were a lot of unknowns, and a few rounds of guess-and-check. One of the key distractions was thinking that each symbol was a letter. That is not the case! From the clues above, we can basically figure out that the process goes like this:

  1. Each letter first gets transformed into a 2 digit number using a Polybius Square (like clue 3).
  2. From there, the 2 digit number (values maxing out at 66) gets transformed to a 6 bit binary number.
  3. The binary number then gets logically XOR’d with the binary numbers produced by the newspaper article. (This is called the pad)
  4. The final binary gets split up into 3 bit chunks which get encoded into the final glyphs we found.

To successfully decode the message, we need to reverse the process. Looking at the clues, I made a guess as to what value the glyph positions indicated. After running the program, I didn’t get meaningful output, so I reversed them, and voila, it worked.

This time, I wrote the program in Python since Python is so much more forgiving with types.

The program first takes the pad text (the newspaper article) and converts it into binary.

I converted the glyphs to digits by hand, which was the most time consuming part. I separated the first digit and second digit into two different arrays. Vertical/horizontal lines are called ‘messageA’ and diagonal lines are called ‘messageB’.

I then convert these values sequentially into binary to create a “messagebin” array. Each value in the array gets XOR’d against the corresponding position’s value in the newspaper binary pad array. The result is processed 3 bits at a time and converted into an array of digits.

The list of digits is processed in pairs, and letters are looked up from a table. There was a clue showing that the table of letters was a clockwise spiral starting from the lower left. By adding digits to the spiral at the end, it worked out to fit perfectly in a 6 by 6 arrangement.

After some finagling, the program successfully deciphered the message! I was then able to thwart the would-be criminals!

def bin(x):
    return ''.join(x & (1 << i) and '1' or '0' for i in range(1,-1,-1))

# Convert pad to binary
vowels = ["a","e","i","o","u","y"]
padbin = []
pad = "The whole-grain goodness of blue chip dividend stocks has its limits. Utility stocks, consumer staples, pipelines, telecoms, and real estate investment trusts have all lost ground over the past month, even while the broader market has been flat. With the bond market signalling an expectation of rising interest rates, the five-year rally for steady blue-chip dividend payers has stalled. Should you be scared if you own a lot of these stocks, either directly or through mutual funds or exchange-traded-funds? David Baskin, president of Baskin Financial Services, has a two-pronged answer: Keep your top-quality dividend stocks, but be prepared to follow his firm's example in trimming holdings in stocks such as TransCanada Corp., Keyera Corp., and Pembina Pipeline Corp. Lets have Mr Baskin run us through his thinking on divedend"

for c in pad:
    if ord(c) < ord('A') or ord(c) > ord('z'): continue
    if c.lower() in vowels :
        padbin.append(1)
    else:
        padbin.append(0)

# Message based on pictograms
#  2 0 3
#   \|/
# 3 - - 1    where straight lines are first digit (A)
#   /|\      and slanted lines are second digit (B)
#  1 2 0

messageA = [2,3,2,2,0,3,3,0,0,2,2,0,3,2,1,3,0,3,0,3,0,0,2,0,3,2,
            2,1,0,2,1,1,2,0,1,0,2,2,2,2,2,3,2,1,2,1,3,2,
            3,3,0,2,2,1,1,1,0,3,2,1,2,0,0,1,3,0,3,2,3,0,1,
            0,1,3,3,2,0,0,0,3,1,1,3,0,1,2,3,1,0,0,3,2,3,1,
            0,0,0,0,3,0,2,2,3,1,0,1,2,0,0,1,3,0,1,1,2,0,2,0,
            2,1,0,3,3,3,2,3,2,0,1,3,3,0,3,1,3,1,0,0,0,0,
            2,2,0,3,2,0,2,2,3,2,0,0,3,2,2,1,3,0]

messageB = [0,3,2,1,0,3,0,1,2,0,2,2,2,0,1,3,3,0,3,2,3,0,2,1,3,3,
            3,0,3,2,3,3,0,1,1,3,2,0,0,0,2,3,0,3,3,3,3,2,
            0,3,1,0,1,0,2,1,0,2,3,3,2,2,0,0,1,2,3,0,1,3,2,
            1,1,3,2,3,2,1,0,2,0,0,0,1,0,3,1,0,2,0,0,3,1,0,
            3,3,1,2,3,2,3,1,0,2,3,2,2,0,3,3,1,0,0,1,1,3,3,2,
            0,3,2,2,0,1,3,3,0,2,2,3,0,0,0,2,0,3,3,1,3,3,
            3,2,2,0,0,3,2,3,2,3,2,2,1,0,3,3,0,2]

# Convert pictogram elements to binary
messagebin = ""
for i in range(len(messageA)):
    messagebin += bin(messageA[i])
    messagebin += bin(messageB[i])

# XOR pad and message together in binary
charcoords = []
tmp = ""
i = 0
while i < len(padbin) and i < len(messagebin):
    tmp += str(padbin[i]^int(messagebin[i]))
    # Set length of binary digits here (Probably 3 digit?)
    if len(tmp) == 3:
        charcoords.append(int(tmp, 2))
        tmp = ""
    i += 1

# Convert message from binary to alphabet based on Polybius square
key = [["F","G","H","I","J","K"],
       ["E","X","Y","Z","0","L"],
       ["D","W","7","8","1","M"],
       ["C","V","6","9","2","N"],
       ["B","U","5","4","3","O"],
       ["A","T","S","R","Q","P"]]

for i in range(0, len(charcoords)-1, 2):
    if charcoords[i] > 5 or charcoords[i+1] > 5:
        print "?",
    else:
        print key[charcoords[i]][charcoords[i+1]],

This was a very enjoyable exercise, and taught me some basics about ciphers (and patience). I hope they put on similar exercises in the future!

Post Apple Announcement Letdown

sadmacMy fanboism toward Apple has been waning for the last few years. Apple does not seem to share my priorities. A few examples:

  • I find no value in the Apple Watch.
  • I think that computing should be done from proper computer, and not an iPad or iPhone.
  • Focusing on the iOS and social integration into Mac OS X instead of more pressing matters (Like a new File System, or simply some bug fixes!) is a mistake.

Misleading The Fans

hello_again_event

This most recent announcement just about seals it though. The advertisement for the event featured the words “hello again.” For loyal Apple customers, this phrase has significant meaning, as it was central to the marketing that marked the rebirth of Apple with the introduction of the iMac by Steve Jobs.

imac-hello-again

When they used this phrase with the iMac, Apple was referencing an even earlier product that marked a significant shift in computing in the 1980’s. The original Macintosh. The first promotional shot of this computer-that-changed-everything had the playful “hello” welcoming user’s to its new graphical user interface.

Apple set up the long-time fans of Apple for something revolutionary. Something that would be amazing, and something most-certainly to do with the iMac, which is frankly Apple’s core product. It is the emblem of Apple and their revitalization in the 90’s. They had to know those words would make us expect an iMac announcement; especially after Microsoft’s announcement the day before, featuring the iMac like surface computer that works like a tablet.

microsoft_surface_studio_all_in_one_desktop_computer_1MacBooks

But instead of updating the iMac (or the MacMini or the MacPro…), they spent the majority of the time lauding the new MacBook Pro computers.

Let’s review all the amazing features that professionals were fervently waiting for:

  • They are thinner! (I’d rather have better heat dissipation, a better graphics card, and more battery capacity)
  • They have more pixels! (But I already have a large display on my desktop that I plug into)
  • There is a touchbar! (I don’t like having to look at what I am touching. Physical keys don’t jump around and can be felt without looking)
  • The track pad is bigger! (Was the previous track pad too small?)

Its not all superfluous though. There are some cool features that are exciting:

  • Touch id for logging in is a cool and novel feature.
  • USB3/Thunderbolt 3 does have interesting prospects since any port can be used for anything, including charging. (As long as they don’t change the port in 2 years and make us buy all new peripherals again…)
  • 2 TB SSD with 3 GB/s read/write speeds

Overall, it felt like a very weak press conference. It’s clear that they just don’t care about the desktop user. They want to appeal to young on-the-go people and their wallets.

Additionally Apple has been slowly killing useful features from their laptops for quite sometime. Here’s a brief list of the once common features that I wish were still present.

  • Battery Indicator Lights – Nice to check battery level without having to open up the laptop, and wait for it to power up
  • Non-soldered RAM – It used to be nice upgrading the RAM down the line, which would be more cost efficient. Admittedly, this isn’t to Apple’s favor though
  • Replaceable Battery – Less of a problem now than it used to be, still would be nice
  • IR Reciever  -Useful for presentation where you don’t want to be worrying about Bluetooth, or wifi connectivity. Also nice for playing movies if you use your Mac as a TV.
  • Sleep Indicator Light – nice to be able to tell is machine is sleeping or powered off
  • Magsafe Adapter – One of my favorite features to disappear. It has saved my laptop many times!
  • Glowing Apple – Not useful per-se, but definitely symbolic of Apple’s soul for the last decade!
  • Startup Chime – Useful for diagnostics, indicates that the hardware is working

No More Displays, No More Desktops?

lg5kmonitor-800x725

Another nail in the coffin for the desktop line is the fact that they endorsed an LG display for use with MacBooks. To me this signals that Apple will shortly discontinue the MacPro and MacMini line of computers. Apple tends not to use other company’s products in promotional shots and advertisements. The only two products without screens no longer have an Apple-branded screen to showcase and buy (and this screen certainly doesn’t exactly scream Apple). I don’t think this bodes well.

Less-Confusing Monikers?

Tangentially, their whole ecosystem has been very clumsily named lately. (Another problem reminiscent of the 1990’s Apple). They have Airs, Pros, Minis, and other dizzying designations that aren’t consistently used. What makes a Pro a Pro? Why was the MacBook smaller than the MacBook Air? Why is the MacMini not called the MacAir?

I propose a simpler, and far less confusing lineup across all products. There would only be 3 designations to differentiate product sizes, and two prefixes to differentiate between computers and devices. Please see the chart below:

proposedlineupConclusion

I hope Apple realizes that the Professional and Enthusiast users are important. For the last 30 years, the enthusiast and pro-sumer market has been Apple’s evangelists. This market would convince friends and family to purchase Apple products. If Apple Alienates these enthusiastic individuals, it could end up hurting their sales in the long run.

As for myself, I just can’t get excited about an Apple product until they release a decent desktop machine.

Illustrator CS5 on El Capitan

Adobe_Illustrator_CS5_icon

I’ve been sticking with Adobe’s CS5 for quite a while now. I had planned on upgrading when CS7 was released… However Adobe decided to go with the cloud-based option, and I’m not ready to have another monthly-fee to worry about.

At any rate, my plan of sticking with CS5 was going just fine until El Capitan came out. 3 (interrelated) problems arose which made using Adobe Illustrator less than ideal.

  1. Adobe Illustrator would crash on exit
  2. Since it crashes on exit, the preferences would never get updated
  3. Since the preferences never get updated, certain settings would need to get changed every time I opened AI.

It was frustrating, but it still wasn’t a deal breaker. I looked online to see if anyone had any solutions. In this post at the official Adobe forums, there was someone asking my exact question. The official response for the thread though was “CS5 is not supported. Update to creative cloud.” Not the answer I was looking for…

As I went deeper, there was another popular suggestion: Downgrade to Yosemite if you are too cheap to upgrade to the newest Illustrator. While that would resolve one problem, it would introduce other issues. Yosemite no longer receives all of the security patches that come with El Capitan. Apple only pushes them out to the newest OS unless it’s super-critical. I’m not willing to give up security for this minor of an issue.

Finally, after wading through multiple pages of this post, a user produced a tip that amazingly solved the issue.

Rename “/Library/Application Support/Adobe/CS5.5ServiceManager” to “/Library/Application Support/Adobe/CS5.5ServiceManager.bak”

This resolved the issue. I’m annoyed that it wasn’t marked as the thread’s solution to the problem, as it is (in my opinion) the best solution to the problem. Illustrator is now working again as it always had before upgrading to El Capitan. I hope that this helps someone else who is having a hard time finding the solution to this annoying issue.

© 2007-2015 Michael Caldwell