diff options
Diffstat (limited to 'tech/set up debian droplet python 3 + gunicorn + supervisor.txt')
| -rw-r--r-- | tech/set up debian droplet python 3 + gunicorn + supervisor.txt | 166 |
1 files changed, 166 insertions, 0 deletions
diff --git a/tech/set up debian droplet python 3 + gunicorn + supervisor.txt b/tech/set up debian droplet python 3 + gunicorn + supervisor.txt new file mode 100644 index 0000000..3f199b9 --- /dev/null +++ b/tech/set up debian droplet python 3 + gunicorn + supervisor.txt @@ -0,0 +1,166 @@ +Set Up Debian Droplet - Python 3 + gunicorn + supervisor + +[reference: +<http://michal.karzynski.pl/blog/2013/06/09/django-nginx-gunicorn-virtualenv-supervisor/> +<http://wiki.nginx.org/HttpHeadersMoreModule#more_clear_input_headers> +<https://github.com/nbs-system/naxsi/wiki/basicsetup> +<http://pillow.readthedocs.org/en/latest/installation.html#linux-installation> +<http://codeinthehole.com/writing/how-to-install-postgis-and-geodjango-on-ubuntu/> +<http://docs.gunicorn.org/en/latest/configure.html> +<udo update-rc.ttp://edvanbeinum.com/how-to-install-and-configure-supervisord/> +] + +If you really want python3.3 you can compile it from scratch. That would eliminate the need to install virtualenv, since it's part of Python as of 3.3. I've gone that route, but for simplicity's sake most of the time I just use python 3.2 which has is available in the debian stable repos. I also grab pip from the repos, though gunicorn and supervisor I install via pip since I want those on a virtualenv-based per-project basis. + +So start with this: + + apt-get install python3.2 python3.2-dev python3-pip + +And then: + + pip-3.2 install virtualenv + +That gets us a nice python3 working setup, though note that you have to call python with python3 and pip-3.2. To cut down on the typing I just make aliases in my .zshrc along the lines of: + + alias p3="python3 " + alias p3p="pip-3.2 " + +Okay so we can use that to setup a working django environment with `virtualenv`. You can use [`virtualenvwrapper`](http://virtualenvwrapper.readthedocs.org/en/latest/) if you like, I find it to be unnecessary. I do something like this: + + mkdir -p apps/mydjangoapp + cd !$ + virtualenv --distribute --python=python3 venv + source venv/bin/activate + +There are few other things that you may want to install before we get around to actually installing stuff with pip. For example if you plan to use memcached you'll want to install pylibmc which needs: + + sudo apt-get install python-dev libmemcached-dev + +**Apparently pylibmc doesn't work with Python3 yet** + +Then I just load everything I need from my requirements.txt file, which lives in the config folder: + + pip install -r config/requirements.txt + +Where did that file come from? Typically I generate it with `pip freeze > requirements.txt` in my local development environment. + +Among the requirements will be gunicorn. If you don't already have a requirements file then you'd just do this: + + pip install gunicorn + +Okay, so we have our sandboxed python3 environment, along with gunicorn to act as a server for our site. In a minute we'll connect Nginx and gunicorn, but first let's make sure our gunicorn server restarts whenever our machine reboots. To do that we'll use `supervisor`. Here's where it gets tricky though, `supervisor` doesn't run under Python 3. It has no problem *managing* python 3 projects, it just doesn't run under python 3 yet. That means we can't just install it using pip3.2. + +We could install it with the system pip, but debian (and ubuntu) have a supervisor repo, so we can just do: + + sudo apt-get install supervisor + +That will install and start supervisor. Let's add an init script so that supervisord starts up should the server need to reboot. so create the file + + /etc/init.d/supervisord + +And grab the appropriate [init script from the supervisor project](https://github.com/Supervisor/initscripts). I use the Debian script from that link. Paste that script into `/etc/init.d/supervisord` and save. Then make it executable: + + sudo chmod +x /etc/init.d/supervisord + +Now, make sure supervisor isn't running: + + supervisorctl shutdown + +And add supervisor to + +With Supervisor installed you can start and watch apps by creating configuration files in the `/etc/supervisor/conf.d` directory. You might do something like this, in, for example, `/etc/supervisor/conf.d/helloworld.conf`: + + [program:helloworld] + command = /home/<username>/apps/mydjangoapp/venv/bin/gunicorn -c /home/<username>/apps/mydjangoapp/config/gunicorn_config.py config.wsgi + directory = /home/<username>/apps/mydjangoapp/ + user = <non-privledged-user> + autostart = true + autorestart = true + stdout_logfile = /var/log/supervisor/helloworld.log + stderr_logfile = /var/log/supervisor/helloworld_err.log + +You'll need to fill in the correct paths based on your server setup, replacing <username> with your username and `mydjangoapp/etc...` with the actual path to the gunicorn app. This also assumes your gunicorn config file lives in `mydjangoapp/config/`. We'll get to that file in a minute. + +First, let's tell supervisor about our new app: + + sudo supervisorctl reread + +You should see a message `helloworld available`. So Supervisor knows about our app, let's actually add it. + + sudo supervisorctl update + +Now you should see a message that says something like `helloworld: added process group`. Supervisor is now aware of our hello world app and will make sure it automatically starts up whenever our server reboots. You can check the status of our gunicorn app with: + + sudo supervisorctl status + +Right now that will generate an error that looks something like this: + + helloworld FATAL can't find command '/home/<username>/apps/mydjangoapp/venv/bin/gunicorn' + + + + + +Now we just need to set up that gunicorn_config.py file we referenced earlier. + +In my setup that file looks like this: + + from os.path import dirname, abspath,join + # get the root folder for this project, which happens to be two folder up + PROJ_ROOT = abspath(dirname(dirname(dirname(__file__))))+'/' + command = join(PROJ_ROOT, "/venv/bin/gunicorn") + pythonpath = PROJ_ROOT + bind = '127.0.0.1:8002' + workers = 3 + log_level = "warning" + error_logfile = "/home/<username>/logs/gunicorn.error.log" + +This is pretty boilerplate, you just need to adjust the paths and it should work. The other thing to note is the line `bind = '127.0.0.1:8002'`. That's the address we'll pass requests to with Nginx. + +Okay, now let's go back to the Nginx tutorial we worked with in the previous part of this series. Here's what this looks like: + + + + # define an upstream server named gunicorn on localhost port 8002 + upstream gunicorn { + server localhost:8002; + } + + server { + listen 80; + server_name mydomain.com; + root /var/www/mydomain.com/; + error_log <path to logs>/mydomain.error.log main; + access_log <path to logs>/mydomain.access.log main; + # See http://wiki.nginx.org/HttpCoreModule#client_max_body_size + client_max_body_size 0; + + # this tries to serve a static file at the requested url + # if no static file is found, it passes the url to gunicorn + try_files $uri @gunicorn; + + # define rules for gunicorn + location @longgunicorn { + # repeated just in case + client_max_body_size 0; + + # proxy to the gunicorn upstream defined above + proxy_pass http://gunicorn; + + # makes sure the URLs don't actually say http://gunicorn + proxy_redirect off; + # If gunicorn takes > 3 minutes to respond, give up + proxy_read_timeout 3m; + + # make sure these HTTP headers are set properly + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + + } + + + } + + + |