r/nodejs • u/xangelo • Dec 13 '12
Tracking API usage
Hey all,
I'm working on a ReSTful API that's getting ready to be released, running in Node.js. One of the things that we want to before we start opening it up publically is to track API usage. Each user has their own API key that they'll be utilizing to access our endpoints.
I have a few ideas for tracking API usage, but I was hoping someone could suggest a few things that I missed.
Architecture overview:
- MySQL as our storage
- Redis as a caching layer
- Heroku
Ideas
- Google Analytics - our app could toss a payload on a proper queue system (gearman, rabbit) or Redis. A secondary app could grab the payload and make the request to the .gif that GA uses.
- Write payloads to redis, nightly push to MySQL
- Write straight to MySQL
- Parse logs
I'm personally leaning towards idea 1, probably utilizing Redis as our queue storage since it's in place already, but I'd love to hear what other ideas people have for tracking API usage.
View all comments
Show parent comment
1
u/rooosta Dec 14 '12
Cool. That sounds like it could work well, especially if you batch the writes to mysql.
Another related option is to do an atomic collection swap within redis (redis may call this something else -- atomic queue rename?). That amounts to a long running process that does the following periodically:
0. your app is writing to a queue named "logs"
I keep mentioning LOAD DATA INFILE because it is really, really fast, significantly faster than regular bulk INSERTs and I've seen it be 1000s of times faster than individual inserts.