Hoorah for standards (and librarians)

Librarians do it with Z3950. Of course, I knew that all along. I just didn’t know what it implied for the non-librarian.  I have been wondering for some time how to get at the bibliographic riches of our national and university library catalogues without using one of the artfully constructed snazzy web interfaces they all seem to have set up in the interests of “usability”. Which interfaces don’t usually include any means of answering such questions as “List dates of publication for all editions of this title which you hold”. BUT (tl:dr) most library catalogues also run a server which exposes their data via an antique API called “Z3950”, originally developed by the Library of Congress and others, apparently so that they could steal catalogue records from each other without the pain of sending round a man with a truck.

When I say antique, believe me I am not joking. Some Z3950 servers I have looked at (eg the Bodleian) will deliver records in funky modern formats like SGML, but the only format you can be sure of getting is US MARC 21. This is an old school binary format, remarkably similar to the ones I used to have to deal with when debugging IDMS databases or alien magnetic tape formats back in the 1970s. You have pointers and variable length fields and a mix of character data and binary and you really don’t want to hack this stuff in Python, honest. Which is why there are dozens of snazzy interfaces and convertors and alternative syntaxes listed in bewildering plenitude on the Library of Congress website. Some of them cost money. Most of them are for Windows only. I took an instant dislike to nearly all of them… until I discovered YAZ.

YAZ is a properly constructed library of utilities offering all sorts of features I don’t claim to understand; what I liked is that it also provides a small number of command line utilities which do things like access a z3950 server, search for records, save the results, and convert from US Marc to something a tad more readable. And it’s open source, and free, and runs on linux, just like that, no need to install lots of bloatware as well. Sudo apt install yaz* and I’m done.

Now, let’s say I want a list of the publication data for all copies of Disraeli’s Sybil held at the British Library. Simple. On the unix command line I type:

$ yaz-client -u xxxxxxxx z3950cat.bl.uk:9909/ZBLACU

(the row of xs there is my authentication string, which I promised the BL I wouldn’t share with anyone as a condition of getting access to their server for nothing. The rest is common knowledge though – see http://www.bl.uk/bibliographic/z3950configuration.html)

The yaz client operates a bit like telnet used to: at each prompt I can type special commands like “find” and “show” and see the results. By default it uses something called Prefix Query Notation, or PQN, though I could also use something called Common Command Language (CCL) “an international standard query language often used in libraries” according to Wikipedia.

Z> find @and @attr 1=1003 disraeli @attr 1=4 sybil

Don’t you just love that PQN syntax? It’s like typing in Latin. It means Find records with “disraeli” in marc field 1003 and “sybil” in marc field 4. Who needs high falutin nonsense like Xquery?

Then I can type:

Z>show 1+100

to see the first 100 records (it’s obvious, surely, that “n+m” must mean “up to m records following record number n”)

Even better, I can type

Z>set_marcdump wibble

and have a copy of  all the marc records returned saved to the file wibble. Why should I do that? Because I can then process that file to something readable with another little command line utility called marcdump:

$yaz-marcdump wibble | grep 204

which will convert wibble to something more readable, and pick out all the lines containing publication data! Yes, you do have to know what those Marc field labels mean, Cynthia.

And yes, there are features for batch mode operation so I could run my commands from a file, save the results, tidy them up and put my own super-simplistic interface on the whole business. I can see this is going to keep me happily occupied for quite a while.

---

---