Disclaimer: Since Khan Academy has migrated to a different system for generating math problems, this post is now out-of-date. However, I'm keeping it around for old times' sake. It was intended mainly for beginners to web development.
Several months ago, I learned about the Khan Academy’s open source Exercises framework. I had never before contributed to an open source project and was pretty new to web development. KA’s Getting Involved section was packed with helpful starting points. It seemed like a great opportunity to learn while helping real users in a very tangible way. So, without further ado, I dove in.
I started first by modifying an existing exercise (a response to a task on the Trello board, labeled green for beginners). After that, I tackled the goal of making a new, interactive exercise. Those two pull requests were pushed live:
- Add problem type to the Slope of a line exercise (my pull request here)
- Create new exercise, Telling time 2 (my pull request here)
The following tips are things I wish I had known/done/used when I was getting started. I hope you can find some/any part of them useful.
1. Reference the wiki and check out Khan Academy's new ReadTheDocs documentation
Back when I got started, I used the wiki extensively, but I hadn't heard of recent KA's ReadTheDocs documentation. It might have not even existed then.
2. Clone the wiki to simplify searching the documentation for obscure features
When starting with the “add problem type” task, I had no idea what a problem type was. I scoured the wiki, but couldn’t seem to surface any details on it. Turns out, it was hidden deep in the wiki, in a sublink at the bottom of the page that I ignored because it looked like the unhelpful “Back to Writing Exercises” link.
May I suggest cloning the wiki to make it easy to search for unfamiliar terms (ex:
data-weight). If you use TextMate (I’m not hardcore with vim yet), command+shift+F for directory search is a big help.
grep is a more universal option. Suggested usage:
grep -n --color “data-type” *, if you’re in the directory.
-n shows the line numbers,
--color highlights the search term. I’m a bit new to grep, so please do comment if you have better solution.
Note: it would be fun to create a nice searching feature for the wiki. Adding it to my todo list as a potential project.
3. If necessary, study source code to know what functionality is/is not available
Maybe this is obvious, but I guess the main point here is that the documentation for some things may not be super mature yet, so you should budget time for studying existing code.
For example, if you’re trying to make an interactive exercise, you might want to examine the HTML source of Creating bar charts 1. After that, you can drill deeper into module source code (ex:
utils/interactive.js) to see if what you need is already built in. The Good examples and pro tips page is a good place to start to find model exercises. If you exhaust that, you can go to town browsing other exercises in the
exercises/ directory of localhost.
I spent a fair bit of time investigating the HTML source of cool interactive exercises, the interactive module (
utils/interactive.js), and the legacy, non-interactive Telling Time code before writing anything. This was particularly important because I realized I’d need to add a feature to interactive.js to allow the clock hands to snap to points around the circle.
4. Use &debug
As stated in the “useful tips” section,
You can append
&debugto the end of the exercise URL (ex: creatingbarcharts.html?seed=144&problem=0&debug) to see the values of the variables used in the problem. In this special debug mode, you can type "h" to reveal the next hint.
If there are no other parameters (ex: “seed” and “problem” in the above example), just append the URL with ?debug (ex: creatingbarcharts.html?debug). For me, the biggest benefit of using this mode is the “h” shortcut. Really beats clicking the hint button ten times in a row.
Note: the debug view only shows variables declared in the
5. Adhere to formatting conventions
Whoops! My previous instructions were all wrong. I apologize. The following style guidelines are straight from the Khan Academy developer docs:
6. Try harder to be clear and friendly in the hints
For Telling Time 2, the thing that took the most feedback/iteration (and time from the busy Khan Academy developers) was the hints section. I struggled to make my original hints simple and clear and ultimately failed. Luckily, @mwahl came to the rescue.
The next time I’m writing hints, I’ll try to imagine that I’m sitting next to a 3rd grader explaining step-by-step how to solve the problem.
7. Check sandcastle, and don’t take it personally if you don’t hear back for a couple of weeks after submitting your pull request.
The team is super busy. For evidence, check out the bugs that users have reported on the Khan Academy site: >1,800 open issues as I write this. Which is not surprising, given they had over 4 million unique visitors in December last year. They have to look closely at your code to make sure you won’t break anything with the site. A single buggy exercise can generate hundreds of issues from users (who are quite vocal about reporting issues when their “streaks” are at risk.
Aside: many of the reported issues are not real bugs (tagged “notabug” by community volunteers).
In the meantime, go back and test your exercise really thoroughly to make sure you’ve ironed out all the bugs you could possibly detect. Some suggestions:
- Have some friends try it out to smoke out spelling/grammar errors and potential usability issues. Push the changes to your forked repo’s branch. The pull request should get updated automatically with the new commits.
- As a first pass, I think the Khan Academy developers will test how your exercise behaves on sandcastle.khanacademy.org. Find your pull request, and click through to access your exercise in the sandcastle environment (a protected version of the Khan Academy site). Make sure it’s working there.
If things get real bad, you could try hanging around the chat room and ask what’s going on.
… Finally, it’s totally worth it.
When I found out that my exercise was pushed live, I was pretty excited. But a few weeks later, I casually checked out the “issues” tab of the khan-exercises Github repo and was met with a flood of user-filed issues related to my exercise. This was definitely cause for alarm, but I mostly floored by the realization that real users were playing with my exercise, on the real Khan Academy site! Plus, the number of users reporting issues is just a fraction of the total, so 10 bugs might mean hundreds of users had given it a spin.
It turned out that almost all of the “bugs” were from users (mostly middle school or younger, judging from the grammar) struggling to properly place the hour hand on the clock. When it’s anything but 0 minutes past the hour, the hour hand falls a fraction of the way towards the next hour, but that’s not super intuitive. People usually place the hand directly on a tick mark with a number. And, at the time of writing, Khan Academy doesn’t yet have a video on how to tell time/set an analog clock.
The exercise is designed in a way that you set the minute and hour hands completely independent of each other. But, in practice, the only time you set an analog clock, there is a single dial which moves both of the hands. Looking back, maybe I should have designed the exercise to mimic setting a real analog clock, or limited the hour hand just to 12 points around the clock. On the other hand, maybe this free-form style more accurately tests students understanding of what the clock hands really represent. The jury still seems to be out on whether or not it should be changed. Ultimately, I think the ability to set an analog clock is pretty useless anyways. =/
(Note: I haven’t yet been able to find an actual technical problem among the reported bugs, but I’ve only been loosely tracking. If you find a technical issue please let me know.)
An important note for those new to web development:
Make sure your browser’s cache is fully disabled.
A simple test for diagnosing if caching is fully disabled
- In an affected file, put an alert in a function you know will run (ex:
- Refresh the page.
If you’re developing in Chrome, I recommend working in incognito mode and disabling cache in Chrome’s Developer Tools (open Developer Tools with command+option+i). Caution: note that this appears to affect all tabs in the mode you’re in. For example, if you click “disable cache” in incognito mode, the checkbox will remain checked within other tabs in “incognito mode”. Thus, you should probably stick with just one mode (incognito) for development, and use the other mode for regular browsing.
If you’re still having troubles and you suspect the browser’s still using cached content, go to the network tab of developer tools, right click, and select “Clear browser cache”.
If you’re interested in dedicating Firefox to development only, type
network.http.use-cache = false browser.cache.offline.enable = false
Boy, did I suffer from some imprudent and hasty searching when I got started. The top Google result for “disable firefox cache” yields this article, which says nothing about the local development case (
browser.cache.offline.enable) - critical when you are running a local server. So I am warning you, and praying that you don’t fall prey to the same mistake. Pun intended.
I hope that some part of this article was useful to you. I’m by no means a super experienced developer, so please excuse my n00bness and correct me where I’ve made mistakes. Please do leave feedback (both positive and negative) and/or questions in the comments!