How To Import a CSV (or TSV) file into a Django Model

How to import a CSV file into Django

Recently I downloaded a US zip code data file in comma separated (CSV) format. Below are the steps and python script that I used to import that data into a django model for my site Wantbox.com.

The Steps to Import CSV Data into Django:

  1. Create your django model (mine is called “ZipCode”, see below).
  2. Create a python script called “load_data.py” with your correct django project and directory info (see mine below).
  3. Put the CSV file and “load_data.py” script in the same directory.
  4. From that directory run: python ./load_data.py”

That’s it. For a CSV file with about 42,000 rows this import took about 15 seconds. You can verify that the data loaded correctly by checking your django admin.

My CSV File

[sourcecode language="python"]ZIPCODE, CITY, STATECODE, STATENAME 02111, BOSTON, MA, MASSACHUSETTS 02481, WELLESLEY HILLS, MA, MASSACHUSETTS 05819, ST. JOHNSBURY, VT, VERMONT etc...[/sourcecode]

My Django “ZipCode” model:

[sourcecode language="python"]import datetime class ZipCode(models.Model): zipcode = models.CharField(max_length=5) city = models.CharField(max_length=64) statecode = models.CharField(max_length=2) statename = models.CharField(max_length=32) create_date = models.DateTimeField(default=datetime.datetime.now) def __unicode__(self): return "%s, %s (%s)" % (self.city, self.statecode, self.zipcode) class Meta: ordering = ['zipcode'][/sourcecode]

My “load_data.py” Python script:

[sourcecode language="python"]# Full path and name to your csv file csv_filepathname="/home/mitch/projects/wantbox.com/wantbox/zips/data/zipcodes.csv" # Full path to your django project directory your_djangoproject_home="/home/mitch/projects/wantbox.com/wantbox/" import sys,os sys.path.append(your_djangoproject_home) os.environ['DJANGO_SETTINGS_MODULE'] = 'settings' from zips.models import ZipCode import csv dataReader = csv.reader(open(csv_filepathname), delimiter=',', quotechar='"') for row in dataReader: if row[0] != 'ZIPCODE': # Ignore the header row, import everything else zipcode = ZipCode() zipcode.zipcode = row[0] zipcode.city = row[1] zipcode.statecode = row[2] zipcode.statename = row[3] zipcode.save()[/sourcecode]

Importing a TSV File (tab separated values) into a Django Model
This script will work for importing Excel TSV files into Django as well. Simply change the python script’s “dataReader” line to this:

[sourcecode language="python"]dataReader = csv.reader(open(csv_filepathname), dialect='excel-tab')[/sourcecode]

About Wantbox
Wantbox is a consumer information website where users publish the things they want and other users supply purchasing recommendations, reviews and typical costs. The service covers a wide variety of consumer and business products and services. On Wantbox you can find someone to remove ice dams, remove a fallen tree limb or help you install new hardwood floors.