Surveying the Remains

There have been 161 TEI council meetings up to February 2023. The minutes of each meeting (conference call or face to face) – except one – are available on the Council website, but only as Word Press pages.

I have tracked down a P4 or P5 source file for 40 of them, covering meetings up to October 2008. I think there must once have been more, because some of the WordPress pages show clear signs of having been adapted or converted from a TEI original. In several cases, some TEI tags are still present, notably <gi> (appears in 20 cases between 2009-04 and 2014-06) or publicationstmt (sic), which appears along with other remnants of a TEI header in 38 cases up to 2016-03. But there is no trace of the original source files anywhere on the current website.

From 2016 onwards, the website provides only Word Press format files, in which HTML tagging is used. However, this tagging is not entirely well-formed: there are many cases where hard line breaks in a table cell are marked by HTML p end-tags, for example. And at least one where the internal structure of a table row has been completely lost.

As a first step, I wrote a perl script which did its best to extract a single well-formed XML document from each set of Word Press pages. This failed consistently for the 36 pre-2016 pages which contain residual TEI tagging but worked reasonably well for the remainder, most of the time. Only 13 of the post-2016 files (out of 85) needed hand-editing to make them well formed, though the tagging still leaves much to be desired. In particular, I realised that some of of the WordPress files made no attempt to preserve the often deeply-nested structure of the minutes, or distinguish marginal annotation from the text.

Since 2016 the minutes have been edited in Google Docs and drafts are therefore (currently) available in Word, ODT, or other formats from the Google Docs website, if you know where to find them. This part (finding them) became much easier when I asked former Council colleagues to share their secret stash of drafts with me. Converting from Google Docs to TEI is comparatively simple and much less error prone than working with the WordPress pages directly. It really ought to be the WordPress pages which constitute the document of record for these minutes, but …

It seemed like a good idea to do a bit of checking in any case. So here’s what I did:

  1. use curl to download all the word press pages to 161 separate files called yyyy-dd.html
  2. use a perl script `articulate.prl` to extract from each of them a (hopefully) well formed xml file containing just the ‘article’ recognised by wordpress; save the result in a file called yyyy-dd_dddd.xml (where dddd is the wordpress article number)
  3. check the well formedness of the resulting files with `xmlwf` and spend no more than a day or two fiddling with the ill-formed ones to improve them
  4. spend a lot of time downloading and renaming files from Google Docs. The downloading was needed for files not in the zip James sent me; the renaming was essential for my personal sanity.
  5. I then enriched the XML file I made in the previous blog entry with links to all the files collected together.

At the last count, there are 162 entries (this includes one which is mysteriously missing from the current TEI website). Of these,

  • 85 are available as well formed wpressxml files
  • 37 are ill formed wpressxml files
  • 41 are only available in a legacy TEI format
  • 115 are available as draft versions from Google Docs

Of the 37 ill-formed word press files, 11 are not also available in Google docs format.

The Google Docs collection lacks anything before 2012-04, and (for no apparent reason) three more recent items : minutes from 2014-01, 2015-10, and 2017-11.

So my next step will probably be to define a target TEI format (with an ODD of course) and set about writing snippets of XSLT.

OpenEdition vous propose de citer ce billet de la manière suivante :
foxglove (6 avril 2023). Surveying the Remains. Foxglove. Consulté le 21 juillet 2024 à l’adresse

Laisser un commentaire

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *

Ce site utilise Akismet pour réduire les indésirables. En savoir plus sur comment les données de vos commentaires sont utilisées.

Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search