Disclaimer: As of 2015, Khan Academy has largely migrated away from its old Exercises framework in favor of the static problem framework, code-named Perseus. This post was intended primarily for beginners to web development. Also, the tips assumed the reader had already gone through the Getting Involved page and gotten set up with the basics of git, etc.

Several months ago, I learned about the Khan Academy’s open source Exercises framework. I had never before contributed to an open source project and was pretty new to web development. KA’s Getting Involved section was packed with helpful starting points. It seemed like a great opportunity to learn while helping real users in a very tangible way. So, without further ado, I dove in.

I started first by modifying an existing exercise (a response to a task on the Trello board, labeled green for beginners). After that, I tackled the goal of making a new, interactive exercise. Those two pull requests were pushed live:

1. Add problem type to the Slope of a line exercise (my pull request here)
2. Create new exercise, Telling time 2 (my pull request here)

The following tips are things I wish I had known/done/used when I was getting started. I hope you can find some/any part of them useful.

1. Reference the wiki and check out Khan Academy’s new ReadTheDocs documentation

Back when I got started, I used the wiki extensively, but I hadn’t heard of recent KA’s ReadTheDocs documentation. It might have not even existed then.

2. Clone the wiki to simplify searching the documentation for obscure features

When starting with the “add problem type” task, I had no idea what a problem type was. I scoured the wiki, but couldn’t seem to surface any details on it. Turns out, it was hidden deep in the wiki, in a sublink at the bottom of the page that I ignored because it looked like the unhelpful “Back to Writing Exercises” link.

May I suggest cloning the wiki to make it easy to search for unfamiliar terms (ex: data-weight). If you use TextMate (I’m not hardcore with vim yet), command+shift+F for directory search is a big help. grep is a more universal option. Suggested usage: grep -n --color “data-type” *, if you’re in the directory. -n shows the line numbers, --color highlights the search term. I’m a bit new to grep, so please do comment if you have better solution.

Note: it would be fun to create a nice searching feature for the wiki. Adding it to my todo list as a potential project.

Directory search in Textmate

3. If necessary, study source code to know what functionality is/is not available

Maybe this is obvious, but I guess the main point here is that the documentation for some things may not be super mature yet, so you should budget time for studying existing code.

For example, if you’re trying to make an interactive exercise, you might want to examine the HTML source of Creating bar charts 1. After that, you can drill deeper into module source code (ex: utils/math.js or utils/interactive.js) to see if what you need is already built in. The Good examples and pro tips page is a good place to start to find model exercises. If you exhaust that, you can go to town browsing other exercises in the exercises/ directory of localhost.

I spent a fair bit of time investigating the HTML source of cool interactive exercises, the interactive module (utils/interactive.js), and the legacy, non-interactive Telling Time code before writing anything. This was particularly important because I realized I’d need to add a feature to interactive.js to allow the clock hands to snap to points around the circle.

4. Use &debug

As stated in the “useful tips” section,

You can append &debug to the end of the exercise URL (ex: creating_bar_charts.html?seed=144&problem=0&debug) to see the values of the variables used in the problem. In this special debug mode, you can type “h” to reveal the next hint.

If there are no other parameters (ex: “seed” and “problem” in the above example), just append the URL with ?debug (ex: creating_bar_charts.html?debug). For me, the biggest benefit of using this mode is the “h” shortcut. Really beats clicking the hint button ten times in a row.

Note: the debug view only shows variables declared in the vars div, so if you’re writing JavaScript you’ll probably still need to use your standard JS debugging methods (breakpoints, console.log, what have you).

Screenshot of debug mode

5. Adhere to formatting conventions

Whoops! My previous instructions were all wrong. I apologize. The following style guidelines are straight from the Khan Academy developer docs:

We generally follow [jQuery's style guide for JavaScript](http://docs.jquery.com/JQuery_Core_Style_Guidelines) with some important exceptions:
* we use 4 space indents instead of tabs.
* we use $ rather than jQuery
* we do not use the liberal spacing with brackets and parens as indicated in the style guide. That is, if-expressions, loop conditions, and functions should follow normal, tight wrapping:

YES:

if (condition) {
    functionCall(bar, ["hi"], function() {
        return 42;
    });
}

NO:

if ( condition ) {
    functionCall( bar, [ "hi" ], function() {
        return 42;
    } );
}

6. Try harder to be clear and friendly in the hints

For Telling Time 2, the thing that took the most feedback/iteration (and time from the busy Khan Academy developers) was the hints section. I struggled to make my original hints simple and clear and ultimately failed. Luckily, @mwahl came to the rescue.

My hints needed reworking

The next time I’m writing hints, I’ll try to imagine that I’m sitting next to a 3rd grader explaining step-by-step how to solve the problem.

7. Check sandcastle, and don’t take it personally if you don’t hear back for a couple of weeks after submitting your pull request.

The team is super busy. For evidence, check out the bugs that users have reported on the Khan Academy site: >1,800 open issues as I write this. Which is not surprising, given they had over 4 million unique visitors in December last year. They have to look closely at your code to make sure you won’t break anything with the site. A single buggy exercise can generate hundreds of issues from users (who are quite vocal about reporting issues when their “streaks” are at risk.

Aside: many of the reported issues are not real bugs (tagged “notabug” by community volunteers).

Some issues on the khan-exercises Github Issues page

In the meantime, go back and test your exercise really thoroughly to make sure you’ve ironed out all the bugs you could possibly detect. Some suggestions:

• Have some friends try it out to smoke out spelling/grammar errors and potential usability issues. Push the changes to your forked repo’s branch. The pull request should get updated automatically with the new commits.
• As a first pass, I think the Khan Academy developers will test how your exercise behaves on sandcastle.khanacademy.org. Find your pull request, and click through to access your exercise in the sandcastle environment (a protected version of the Khan Academy site). Make sure it’s working there.

Screenshot of Sandcastle

If things get real bad, you could try hanging around the chat room and ask what’s going on.

… Finally, it’s totally worth it.

When I found out that my exercise was pushed live, I was pretty excited. But a few weeks later, I casually checked out the “issues” tab of the khan-exercises Github repo and was met with a flood of user-filed issues related to my exercise. This was definitely cause for alarm, but I mostly floored by the realization that real users were playing with my exercise, on the real Khan Academy site! Plus, the number of users reporting issues is just a fraction of the total, so 10 bugs might mean hundreds of users had given it a spin.

It turned out that almost all of the “bugs” were from users (mostly middle school or younger, judging from the grammar) struggling to properly place the hour hand on the clock. When it’s anything but 0 minutes past the hour, the hour hand falls a fraction of the way towards the next hour, but that’s not super intuitive. People usually place the hand directly on a tick mark with a number. And, at the time of writing, Khan Academy doesn’t yet have a video on how to tell time/set an analog clock.

A user-submitted issue

The exercise is designed in a way that you set the minute and hour hands completely independent of each other. But, in practice, the only time you set an analog clock, there is a single dial which moves both of the hands. Looking back, maybe I should have designed the exercise to mimic setting a real analog clock, or limited the hour hand just to 12 points around the clock. On the other hand, maybe this free-form style more accurately tests students understanding of what the clock hands really represent. The jury still seems to be out on whether or not it should be changed. Ultimately, I think the ability to set an analog clock is pretty useless anyways. =/

Telling Time 2

(Note: I haven’t yet been able to find an actual technical problem among the reported bugs, but I’ve only been loosely tracking. If you find a technical issue please let me know.)

An important note for those new to web development:

Make sure your browser’s cache is fully disabled.

This isn’t specific to creating a Khan Academy exercise (goes for any localhost development), but it’s easily the most important thing to check off your list before you get into any JavaScript-ing. If you don’t get this right, changes to local files may not show up when you refresh. You may start thinking your code is buggy when the real problem is that the updated file didn’t get pulled in by the browser.

A simple test for diagnosing if caching is fully disabled

1. In an affected file, put an alert in a function you know will run (ex: alert("show me!");).
2. Refresh the page.
3. If the alert does not result in a pop-up dialog, caching is not fully disabled (or JavaScript is disabled… which you should undo pronto).

Chrome

If you’re developing in Chrome, I recommend working in incognito mode and disabling cache in Chrome’s Developer Tools (open Developer Tools with command+option+i). Caution: note that this appears to affect all tabs in the mode you’re in. For example, if you click “disable cache” in incognito mode, the checkbox will remain checked within other tabs in “incognito mode”. Thus, you should probably stick with just one mode (incognito) for development, and use the other mode for regular browsing.

If you’re still having troubles and you suspect the browser’s still using cached content, go to the network tab of developer tools, right click, and select “Clear browser cache”.

Firefox

If you’re interested in dedicating Firefox to development only, type about:config in your address bar and enter. Then, filter for “cache”. Configure both of the following parameters to false (source).

network.http.use-cache = false
browser.cache.offline.enable = false

Boy, did I suffer from some imprudent and hasty searching when I got started. The top Google result for “disable firefox cache” yields this article, which says nothing about the local development case (browser.cache.offline.enable) - critical when you are running a local server. So I am warning you, and praying that you don’t fall prey to the same mistake. Pun intended.

Conclusion

I hope that some part of this article was useful to you. I’m by no means a super experienced developer, so please excuse my n00bness and correct me where I’ve made mistakes. Please do leave feedback (both positive and negative) and/or questions in the comments!