2010/08/18-19 - goodbye cataloging, hello serials & monographs

The staff meeting Wednesday morning went very well. Aaron, Paula, and Helen presented their plans to merge the acquisitions and cataloging groups into a single unit. Those of us in cataloging will move to new cubes in the acquisitions area of the building by the end of the year. The new unit will be managed as two teams - a monographs team will report to Helen, and Paula will lead the serials and electronic resources team. We all expect our monographs work will continue to shrink over time, but combining acquisitions and cataloging is a good step in streamlining library technical services.

Otherwise I've spent most of the last couple days working on a few patches to the vufind tools. I spent most of the time tracking down the reason why our vufind server does not properly filter connecting diacritics so that a search for vtoriaa finds Russian language records like this. I eventually discovered that vufind's underlying Solr server has a filter system that handles that kind of thing, and that the unicode filter that indexes versions of unicode tokens without diacritics was not properly handling combining halfmark characters. I built an AuUnicodeFilter copy of the unicode filter that just adds a switch-statement block that checks for the combining half marks; problem solved!

2010/08/16-17 - patching code

I've spent the last several days on a series of small tasks that added up to take all my time. I checked in several patches to small bugs in the AuCataloging tools, worked on the ACES-project metadata import tool, setup a d-space collection on repo for a professor in computer science to experiment with, installed a valid SSL certificate on repo, .... stuff like that.

Tomorrow there's a big meeting for everyone in the systems, acquisitions, and cataloging groups where Aaron will present some vision of the future of technical services. I'm looking forward to it - technical services can use an overhaul. The cataloging group shrank a lot over the last year with Harriet retiring, Henry's retirement and death, Tom's unexpected death, and Lori leaving for a new job. Then a few weeks ago three women in cataloging were told out of the blue that they would begin reporting to the circulation group. The move was perceived by staff as a ham handed decree from library leadership, and one of the transferred women was angry enough to go ahead and retire rather than accept the change. I'm sure more personnel moves are in store in the near future considering the libraries' budget constraints, but a meeting to present and exchange ideas with staff will help avoid bad feelings.

2010/08/09-10 tweaking vufind

I'm finally back at the library after three weeks away. Three weeks sounds like a long time, but it went fast.

I spent most of today and yesterday experimenting with the Solr index that backs our vufind catalog. Last week the library finally decided to make vufind our default online catalog rather than the older Aubiecat Voyager OPAC.

A few small vufind bugs have popped up that we can deal with, but one big problem we've had is that vufind's Solr server would periodically run out of memory, and require a restart. We've tried several things over the past week (I lent some help from home last week) including moving Solr to a 64 bit Solr and allocating a 5 GB heap, and we also tweaked the Solr cache configuration, but the memory problem persisted.

Last night Clint noticed that the Solr memory use spiked when he ran a title-sort on a search result, so we've been looking at sorting since then. It turns out that Solr's lucene index engine uses a "field cache" to implement sorting, and the cache size is proportional to the number of unique entries in the sort field and the size of each entry. Our catalog has over three million unique titles, so it's very expensive to process a title search. I experimented with a solution that just post-sorts the first part of a relevance ordered search outside Solr, but Clint decided to just disable the title-sort as it's not a critical feature. Problem solved - hopefully!

2010/07/13-15 - ACES Voyager merge pain

I've been working on the tool to decorate the dublin-core for each pdf in the ACES scanning project with metadata from our Voyager catalog. I've got code running that extracts each record's metadata from the mets files the scanning contractor delivered, and also have the list of Voyager ids, so now I need a strategy to match A with B, and glue it all together.

I'm making the best of this ACES project by taking the opportunity to update some shared code and trying some new things, but I believe that the index of the pdf's full-text makes the meta-data from the catalog superfluous for discovery - especially after Google and Bing scan our sever. The librarians refuse to believe that though!

2010/07/08,12 - Honors Theses are Go!

I think the new repo server is nearly ready to start accepting Honors College theses this fall. I updated the site to include links to Midge's final liability release form; submitted a couple test records; spoke with Tony about the web design; sent the following e-mail to Kathie at the Honors College, and finally submitted a purchase order for a new SSL certificate for the server. Tony had the good idea to setup a rule that routes http://repo.lib.auburn.edu/honors/ to the collection too.

--- Reuben Pasquini 7/8/2010 4:58 PM ---
Hi Kathie!

I just want to give you an update on the 
online collection for honors theses.
I think we're on track for a fall release.

*. The attached Word document has the final legal
    disclaimer approved by the lawyers.
    The form is also online:
        http://repo.lib.auburn.edu/HonorsLiabilityForm.pdf 
    and
        http://repo.lib.auburn.edu/HonorsLiabilityForm.doc 
   Let us know if you have questions or see problems.
    
*. The theses collection is at:
         http://repo.lib.auburn.edu/honors/ 
    You can add that link to the Honors College web site
    once we go live.
    We'll remove the 2 test records before then.

*. The library web designer, Tony Oravet, plans to update the
    site design in the next month.
    Please let us know if you have any requests.

*. I'll be out of town July 19 until August 9.
    Can we plan to meet sometime the week of August 9
    to review the site, and to give you some training
    on how to add a student to the system, and how
    to review a student's submission ?
    Monday to Thursday 
    starting 9:30am to 2:00pm is best for me ...

I think that's all I wanted to report.
What do you think ?

Cheers,
Reuben

2010/07/06-07 - vufind redux

I spent most of my time the last couple days patching a couple search related bugs that pop up in corner cases - one with "author" search and another with very long search strings. I think the patches are ok, but there's no regression test suite, so we'll see.

I need to get back to work on the AU-repo ACES import and configuration for the Honors College collection. I made a few small changes yesterday, but I'll do more tomorrow.

06/29-30,07/01/2010 - respect cataloging!

Over the last few days I managed to stage Claudine's ACES collection scans on our server, configure an XSL transform to translate each scan's title, author, date metadata file to the format Dspace wants for bulk import. I imported some records into a test collection , and sent an e-mail out to let several people take a look. The librarians want to enrich the Dspace records with metadata from our Voyager catalog. For example, this Dspace record corresponds to this Voyager record. I pointed out that the DSpace server indexes the OCR full-text of each record's PDF content, so the extra metadata won't help much with discovery. Librarians don't like it when you say things like that!

I spent most of Thursday preparing a patch that fixed a bug Julie at the graduate school found in the ETD proquest-export tool.