circulation data, part seven

okay, importing the circulation data was far harder than I wanted it to be. I’m still not sure that I did it right – or even in the easiest way.

Circulation data, at the beginning, is the records which say this barcode id is checked out to this patron id on this date – fines data would generally be included but we don’t issue fines in our library. Theprocedure is to export it from Athena; edit it if necessary (like the marc data); import it into Koha; check it.

It was a little fiddly to export the correct parts of the circulation data from Athena; Athena doesn’t give you a lot of options, and even finding the exported files was a little awkward! It produced a series of dbf and asp files; one of these dbf files was the one I wanted – but it wasn’t labelled anything useful like “circulation data”!

I made the decision to ditch all the circulation history data while exporting from Athena; it wasn’t as important to me to know how many times an item had been borrowed as it was for the migration to run quickly. I accept that this data is useful for weeding purposes, but having done a really thorough weed immediately before the migrate, I figured the items on the shelf were consistent enough with each other. Also; I still have date stamps on the inside of the book covers. I am old school. So the amount of times a book has been borrowed before Koha is stamped right there in the book.

I used the koha offline circulation uploader tool to import the circ data. The wiki outlines that offline circ data should be in a tab delimited csv file, with columns for date – action – patronid – barcode. I sorted the circ data I had by action, discarding actions like lost, missing, or withdraw, leaving me with “check out” or “renew” – both of which I changed to “issue”. Patronid data and barcode id already in place, I reformatted the date to match Koha’s preferences and copied the first four rows as a test.

A test which worked! First go! But this was the last time it would work first go, sadly.

I found that trying to import more than forty records in any batch tended to throw a bunch of errors, or else hang, and not tell you if it had imported correctly. It was easy (though tedious) to copy a screenful of records to a new sheet, save that as “oc import 1.koc”; upload it; while waiting for the upload save the nex bunch in another new sheet “oc import 2.koc”; see if any items hadn’t worked, and copy those records to a text file called “errors jan”; upload the next file; save the next bunch of records as “oc import 3.koc”….

Perversely, finding errors was a good thing – cause this meant it had uploaded the data properly. More often, at least at the beginning of the process, it would choke and hang, and never come back with any kind of results. I’m pretty sure I ended uploading oc import 3.koc nine or ten times before it finally worked!

The errors it found, in most of the cases, were because I hadn’t actually managed to import one batch of marc records at all. In others, as far as I could tell, it recorded an error just because the program wanted to make me sad. Thinking about it now, a week later, I am very glad that I got everyone to return as many items as possible over the summer; I had about 1800 records to import, in all. I want to find an easier way to do this part, cause how could you migrate a system with 20,000 records if you had to double check 40% of them?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s