Python | Mitch Fournier

Tag: python

How To Import a CSV (or TSV) file into a Django Model

How to import a CSV file into Django

Recently I downloaded a US zip code data file in comma separated (CSV) format. Below are the steps and python script that I used to import that data into a django model for my site Wantbox.com.

The Steps to Import CSV Data into Django:

  1. Create your django model (mine is called “ZipCode”, see below).
  2. Create a python script called “load_data.py” with your correct django project and directory info (see mine below).
  3. Put the CSV file and “load_data.py” script in the same directory.
  4. From that directory run: python ./load_data.py”

That’s it. For a CSV file with about 42,000 rows this import took about 15 seconds. You can verify that the data loaded correctly by checking your django admin.

Continue reading »

Basic Python Logging in Django 1.2: Debugging Made Simple

When I was deploying my new social shopping site Wantbox a couple of month ago, I discovered that I couldn’t use the Django print command in my live Dreamhost (Passenger WSGI) environment like this:

   print "Hello world!"

As a new site, I was still debugging a few persistent nits and really needed to print out messages when certain areas of the code were accessed. A few sites recommended that I try the Django debug toolbar, but I was having trouble getting that to work reliably in my Dreamhost setup.

I searched around a bit more and after consuming many web pages, finally discovered a dead simple solution using the built-in Python logging module.

Continue reading »

Show a Custom 403 Forbidden Error Page in Django

Creating a custom 404 Page Not Found error page is so easy in Django (all you do is put your own template named “404.html” at the root of your templates directory) that I naturally assumed doing the same for a 403 Forbidden error page would be just as easy. Unfortunately it is not.

After searching around for quite a while last night, I found bits and pieces that I have modified slightly and republished below in a unambiguous step-by-step tutorial (see the “Source and Other Resources” section at the end of the post for a few of the source posts).

The method posted below leverages some custom Django middleware code. Please, if you have a better, more elegant solution I’d love to hear about it in the comments.

Create the Middleware

  1. Create a directory at the root of your project called “middleware”
  2. Add a file named “__init__.py” to this directory
  3. Create a file named “http.py” in this directory with the following contents:
  4. from django.conf import settings
    from django.http import HttpResponseForbidden
    from django.template import RequestContext,Template,loader,TemplateDoesNotExist
    from django.utils.importlib import import_module
    
    """
    # Middleware to allow the display of a 403.html template when a
    # 403 error is raised.
    """
    
    class Http403(Exception):
        pass
    
    class Http403Middleware(object):
        def process_exception(self, request, exception):
            from http import Http403
    
            if not isinstance(exception, Http403):
                # Return None so django doesn't re-raise the exception
                return None
    
            try:
                # Handle import error but allow any type error from view
                callback = getattr(import_module(settings.ROOT_URLCONF),'handler403')
                return callback(request,exception)
            except (ImportError,AttributeError):
                # Try to get a 403 template
                try:
                    # First look for a user-defined template named "403.html"
                    t = loader.get_template('403.html')
                except TemplateDoesNotExist:
                    # If a template doesn't exist in the projct, use the following hardcoded template
                    t = Template("""{% load i18n %}
                     <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
                            "http://www.w3.org/TR/html4/strict.dtd">
                     <html>
                     <head>
                         <title>{% trans "403 ERROR: Access denied" %}</title>
                     </head>
                     <body>
                         <h1>{% trans "Access Denied (403)" %}</h1>
                         {% trans "We're sorry, but you are not authorized to view this page." %}
                     </body>
                     </html>""")
    
                # Now use context and render template
                c = RequestContext(request, {
                      'message': exception.message
                 })
    
                return HttpResponseForbidden(t.render(c))
    
  5. You should now have this:
    • /myproject/middleware/
      • __init__.py
      • http.py

Modify Your Project’s “settings.py”

  1. Add ” ‘myproject.middleware.http.Http403Middleware’, ” to your MIDDLEWARE_CLASSES

Create a Custom “403.html” page

  1. Put it at the root of your template directory
  2. Sample content: (note: assumes you’ve already defined a “base.html” template)
{% extends "base.html" %}
{% block title %} | Access Denied{% endblock %}

{% block content %}
<h1>Access Denied</h1>
<span>We're sorry, but you are not authorized to view this page (Error: 403)</span>
{% endblock content %}

Raise a 403 Error

  1. In the file where you want to raise the 403 add this at the top: (I used it in my project’s “view.py” file)
  2. from myproject.middleware.http import Http403
  3. Raise the 403
  4. if request.user.id != object.user.id:
        raise Http403

That’s it, you now have a Django template to handle 403 Forbidden errors. I’m sure there’s a way for your front-end, production web server to do the same, but I haven’t explored that yet.

Source and Other Resources

  1. A middleware solution is here (HT Felipe ‘chronos’ Prenholato)
  2. A middleware solution is here (HT Glen Zangirolami)
  3. A potential decorator solution is here (HT Magus)

Getting Started with virtualenv (Isolated Python Environments)

Like South, virtualenv is a helper utility that I put off using for too long. Looking back, it is so easy to get up and running (just like South, see below) that there is no reason for you to hold off like I did.

In a nutshell, virtualenv is a tool for creating isolated Python environments. This is particularly useful if you host multiple Django projects on a single dev box. As an example, virtualenv allows you to easily work on one site built on Django 1.1 and django-registration 0.7  and another one built on Django 1.2 with django-registration 0.8.

It is also invaluable if you want to deploy a Django project to a shared host where you don’t have root access to the main “site-packages” directory. Once you create a virtualenv for your project, an isolated copy of Python and “site-packages” is created which you own and can write to.

Basic virtualenv Start-up Steps

  1. sudo pip install virtualenv
    (or, sudo easy_install virtualenv if you don’t use pip)
    (or, easy_install --install-dir ~/site-packages/ virtualenv on a shared host)
  2. mkdir ~/virtualenvs   (a directory for your isolated environments)
  3. virtualenv ~/virtualenvs/mysite.com --no-site-packages
    (--no-site-packages isolates your environment from the main site-packages directory)
  4. cd ~/virtualenvs/mysite.com/bin
  5. source activate  (activates your new environment)

That’s it, you now have a dedicated Python environment for your mysite.com project with it’s own “site-packages” directory (~/virtualenvs/mysite.com/lib/python2.5/site-packages/) where you can install any version of Django or Django app without messing with your other projects. To exit your virtualenv just type “deactivate”.

A helper alias:

  1. vi ~/.bash_aliases
    alias ams='source ~/virtualenvs/mysite.com/bin/activate'
    
  2. source ~/.bash_aliases  (to activate the aliases)

Now you can run “ams” to quickly activate your “mysite.com” environment. For more in-depth information, check out the virtualenv documentation or a couple other informative blog posts.

Debian 5
Python 2.5.2
virtualenv 1.4.9

Web Analytics