When I started my batch at the Recurse Center, I didn’t know what an API was. I thought this was a problem — I believed the fact that I'd gotten this far without “getting” APIs meant I wasn't cut out for this whole programming thing.
I was so wrong! And thank goodness, because programming is pretty fun and now I get paid to do it.
Tons of programmers don't know what an API is. And before someone knows something, they don't know it! That's just always true.
And, as I found out later, I didn’t need to understand the whole internet before using it to talk to an API.
To build a house, I tried growing trees
After a few days in batch, I'd done enough digging and coding to identify my main problem, and it wasn't that I didn't understand what an API was. A REST API, the kind of API I was focused on, is just a way for programs to talk to each other over the internet using HTTP — at its core, it's mechanically identical to visiting a website in your browser. The real problem was that I didn't understand how to talk to an API. I didn't understand HTTP.
Now that I'd right-sized my learning goals, the next step was much clearer: learn to talk to an API by writing code that talks to an API.
I didn't do that.
Instead I said, Hey, I've got three months! That's basically infinity. If I really want to understand HTTP, I should implement a different application layer protocol from scratch myself. What a great idea! Then I'll understand HTTP, TCP, and whatever other protocol I implement!
So I set out to do this. I was going to write an XMPP library in Go and then use it in a command-line chat client (which I would also write). Each day, I came in to RC early to listen to loud techno, draw complicated diagrams on whiteboards, and read RFCs. The month-long saga plays out in the commit messages of both repos since I was using them as a journal.
Only with the help of other Recursers did I escape this incredible yak-shave. I attended a Saturday hackathon, and another attendee announced she was going to work on a pair programming chat bot called Pairing Bot, and that she'd love help.
While pairing on Pairing Bot for the rest of the day, I had a series of realizations:
- This was fun. I had not been having fun.
- I was learning things that were properly scoped for my experience level. I was learning from my XMPP project too, but I was also grasping in the dark a lot.
- This was a project I could finish during batch, and I really wanted to never graduate with "done" software to show for it.
- Everyone at the hackathon was super excited about our project. Recursers really wanted a pairing bot!
My friends reminded me I could start with the house part
If I hadn't gone to that hackathon, I probably would’ve worked on my ill-fated XMPP library for the rest of batch. But because I went, and because I was vulnerable about my fears with other Recursers, I got to hear exactly what I needed in that moment.
You can just work on something else. That's not failure!
It sounds like you've learned a lot. Taking that knowledge to another project isn't a loss!
It sounds like you want to make something people will use. People will definitely use a pairing bot!
My pairing partner that day finished her batch just a few days later, and gave me her blessing to continue working on Pairing Bot — so that's what I spent the rest of the batch doing. Now Pairing Bot runs in production, she’s used by lots of Recursers every day to find partners to pair program with, and right there in her source code is my first ever API call. Let’s do a close read of it now!
When you consume APIs
// This code is part of a function that runs once per day, and sends PMs to
// each pair of recursers who've been matched for pair programming.
const zulipAPIURL = "https://recurse.zulipchat.com/api/v1/messages"
const matchedMessage = "Morning y'all! You've been matched for pairing today :)\n\nHave fun!"
// Here we're creating a thing that can speak HTTP for us.
zulipClient := &http.Client{}
// This loop iterates over an array of recursers and
// sends a Zulip message to each consecutive pair.
for i := 0; i < len(recursersList); i += 2 {
messageRequest := url.Values{}
messageRequest.Add("type", "private")
messageRequest.Add("to", recursersList[i]["email"].(string) + ", " + recursersList[i+1]["email"].(string))
messageRequest.Add("content", matchedMessage)
req, err := http.NewRequest("POST", zulipAPIURL, strings.NewReader(messageRequest.Encode()))
req.SetBasicAuth(botUsername, botPassword)
req.Header.Set("content-type", "application/x-www-form-urlencoded")
resp, err := zulipClient.Do(req)
// There's some more code here, but we're
// skipping it for the sake of example.
}
Wow! OK that's a lot of stuff. Before we take this line-by-line, there are a few things worth noting:
- Pairing Bot runs on Zulip, which is the chat platform that the Recurse Center community uses.
- Pairing Bot uses the Go HTTP standard library to talk to the Zulip API. It’s common to use a helper library for this sort of thing, but I wanted to deal with the details.
Let’s dig in.
messageRequest := url.Values{}
messageRequest.Add("type", "private")
messageRequest.Add("to", recursersList[i]["email"].(string)+ ", " + recursersList[i+1]["email"].(string))
messageRequest.Add("content", matchedMessage)
Here, we're gathering all the information that Zulip needs about the messages we're sending, and we're packaging it into a group of query parameters called messageRequest. messageRequest is just a Go map with some extra methods that are helpful for this use case.
Zulip needs to know:
- What
type of message we're sending (a private message)
- Who the message should go
to (the next two Recursers in our array)
- What the
content of the message is (matchedMessage)
req, err := http.NewRequest("POST", zulipAPIURL, strings.NewReader(messageRequest.Encode()))
Here, we're creating a new HTTP request called req that we'll eventually send to the Zulip API.
http.NewRequest() needs to know:
- What kind of HTTP request we're sending (POST)
- Where the request should be sent (
zulipAPIURL)
- What the contents of the body of the request are
For us, the body of our HTTP request is just everything we've packaged into messageRequest. But we can't give messageRequest to http.NewRequest(), because messageRequest is a fancy map but http.NewRequest() will only accept a Reader. Fine. I don't quite understand all that, but accepting that not all things must be completely understood, let's just give http.NewRequest a strings.NewReader(messageRequest)? Gah! That doesn't work either, because messageRequest is a fancy map but strings.NewReader() needs a string! OK, it looks like messageRequest.Encode() will give us a URL encoded version of messageRequest as a string.
Finally, we end up giving http.NewRequest() a strings.NewReader(messageRequest.Encode()), and then everything works.
Reality check: While I was in batch, it took me a few days to figure out all that stuff — and as you can see, I still don't totally understand it!
req.SetBasicAuth(botUsername, botPassword)
req.Header.Set("content-type", "application/x-www-form-urlencoded")
So we’ve packaged our HTTP request into req, but we need to do two more things before sending it or our request won't work:
- We need to add Pairing Bot's API access credentials to the HTTP request. We do that using
req.SetBasicAuth() to set the request's Authorization header.
- We need to set the request's Content-Type header to
application/x-www-form-urlencoded so the Zulip API knows how the body of the request is formatted.
resp, err := zulipClient.Do(req)
We send our HTTP request to the Zulip API! The outcome of this API call is that the Pairing Bot Zulip account sends a group PM to a pair of Recursers, telling them that they've been matched for pair programming today.
Write code, delete shame
After getting this API call to work, I called Pairing Bot done and shipped her. There were some bumps straight out of the gate, but we resolved them quickly and soon real people were using her! I was thrilled.
I'd learned some truly new things, and along the way made useful software with positive human impact. Is Pairing Bot’s code good? In my opinion, not really :) — and that’s totally OK! Am I excited to rewrite her in Python using Zulip’s super friendly bot framework? You bet! But the process mattered, and I struck a good balance between choosing tools that made me deal with the details and asking for help staying on track.
Looking back, I’m thankful I started tinkering one layer lower than I needed to. I learned a lot in my XMPP rabbit-hole — most importantly that abstractions are there for a reason. You don’t need to understand all the intricacies of HTTP to use an API, just like you don’t need to grow trees to build a house.
But I’m more thankful I was surrounded by other friendly programmers ready to hand me a ladder, and help me chart a better course. I felt the best — and learned the most — when I stopped reading RFCs, started writing code, and kept asking for help.
Maren Beam attended RC in 2019. Find her online at marenbeam.net.
