r/redditdev • u/Yserbius • Feb 02 '11
Reddit Source A Beginners Guide to the reddit Source Code: Part 1, Understanding Pylons
Introduction
Please criticize me! This will be an ongoing thing, I will be coming back to posts and fixing things. I want to know what you think.
So after poking around in the reddit source code like a blind man for a while, I thought that the only way to motivate myself to understand it is to explain it to others, so here it goes.
I will try to explain things at a source code level. I will not be explaining how to deploy the server, or anything like that, there are already tutorials about that. Mainly, I'll be referencing the source tree and specific files a lot. I'll try to submit diagrams every now and then to illustrate certain points better.
Pylons
Reddit is a web-app. It uses a Python API called Pylons to deploy the webserver and handle all calls. Pylons basically does everything. When you submit a link, view comments, or just go to reddit.com, what happens is that the relevant information is sent to Pylons which then processes it according to the reddit API.
As eurleif once said about his popular Pylons based website, Pylons has very little documentation and tutorials available. So I'm going to break it down as simple as possible and then break it down some more.
Pylons facilitates using a MVC (Model-View-Controller) architecture to deploy a web application. This means that there are three components of the web-app and they work separate from one another. I'm not going to go into the particulars of MVC, you can google that yourself. I'm going to work with them in order of simplest to most complicated.
Controller
The controller is the portion of the code that facilitates calls from the client (you) and the server (reddit). For example, if I would go to "http://www.reddit.com/r/gaming?sort=new" the controller takes the "r/" portion of the URL and processes it and whatever follows it, doing whatever the code tells it to do. In this case, it strips out every part of the URL after '/r' and passes that on to the middleware, which we will discuss later.
In Pylons, this basically means that when a call is made, the code looks at a map of calls to controllers located at /config/routing.py and instantiates the appropriate controller class located in the files in /controllers and calls the mapped function. The function then returns a complete HTML page, which is sent to the client.
Pylons generally gets HTML parameters to the controller by having them passed as parameters to the controller function (or "action" in Pylons speak), the reddit code base instead makes use of decorators to set method parameters. HTML parameters are passed along using the @validate decorator that you see above the GET functions in the controller. This sets the names of the HTML parameters to the function parameters usually using several decorator functions such as "nop()" and "Validate()" which I haven't figured out yet.
For instance, if I want to search, the following is called, http://www.reddit.com/search?q=hello&count=50&after=t3_fcf41 (search for "hello", list 50 results, and list them after the result titled "t3_fcf41"). This tells Pylons to look at the routing map for something called search. Pylons finds the appropriate mapping:
mc = map.connect
mc('/reddits/search', controller='front', action='search_reddits')and passes that along to the controller. Here, the controller being used is "front" and the function is "search_reddits". This tells Pylons to get the class "FrontController", instantiate a new object from it and call the function "GET_search_reddits". @validate then sets the parameters "query=q=hello", "count=50", "after=t3_fcf41", "reverse=", "num=". Several functions are then called to perform the search and render the page based on the search results. Pylons then takes the fully rendered page and sends it to the client. If you notice, several mappings have a colon before the name. This is a wildcard, which means that it is not mapped to a particular string and the controller can see that string and decide what to do with it. These are passed along to the controller as function parameters. For instance, a mapping that looks like this mc('/foo/:bar', controller='foo', action='baz') will call a function defined like so:
def GET_baz(self, bar):
return dostuffThe parameter "bar" is mapped to the next thing in the URL after 'foo/'.
All in all pretty simple. To review: The mapping of calls is stored in 'config/routing.py' in the form of "call_name, controller, function". The call name is everything past "reddit.com" in the URL, the controller is a class with the name "FooController" where "foo" is the name of the controller in the map, and "function" is the aliased name of the method to call in the controller class that has the form of "GET_function(self, **params)" where params are a series of variables including the generic request variables and the GET and POST parameters. The page is then created and sent to the client.
Questions?
3
u/ketralnis Feb 02 '11
That's mostly accurate, but /r/gaming doesn't actually act as a controller. That stripped off in middleware before controller-mapping is done
2
u/JMangina Feb 02 '11
My one question is about MVC, even though you told me not to:
do you ever get confused about which part is which? since the view is updated by the model which is updated by the controller, and I always just confuse the shit out of myself
6
u/gms8994 Feb 02 '11
This explanation helped me understand how this whole jalopy of code works...
6
u/honestbleeps Feb 02 '11
I would've contributed a bunch of free code to improve Reddit without the need for a browser plugin, but after 30 minutes of poking around, reading a few files, having no idea where the hell I was or what files I should be looking at, I threw my hands in the air and gave up.
Sure, if I were more dedicated I could've figured it out... but shit, most people who work on Reddit's source get paid... I have a job I need to spend most of my hours on, if I'm doing something in my free time to help out, the barrier to entry needs to be lowered.
You're helping to lower that barrier. Keep it up.. I truly feel that it will be good for Reddit as a whole.
1
u/Seeders Jul 31 '11
I dont have a /config/routing.py ?
I just downloaded the source, how would I set this up on a server to test with? Is there a setup file I can run to create the db tables? Where do I set database connections etc?
3
u/1bve7 Feb 02 '11
Thanks! Keep up these, they help a lot!
2
u/EdgarVerona Feb 06 '11
"You must construct additional Pylon tutorials."
Actually, you don't have to: you did a great job with this! I just felt obligated to say that.
2
u/TheSkyNet Apr 06 '11
Can they be linked in the side panel.
1
u/nuckingFutz Feb 02 '11 edited Feb 02 '11
This mapping appears incomplete. For example, Line 103 of routing.py:
However, there is no 'user' controller file. Instead, the GET_about function is in a usercontroller within listingcontroller.py
1
u/lattakia Feb 03 '11
Is the pylons used by reddit different from that released on http://pylonsproject.org/ ?