Tuesday, November 23, 2010

2010/11/15-18,22-23 - Happy Turkey Day!

It has been a good couple weeks. I've managed to stay out of trouble, and everyone is looking forward to Thanksgiving. I also managed to get a few things done. First, we imported Claudine's pamphlet collection into our DSpace repository. Jon installed the Solaris freeware xpdf package on repo, which allowed me to enable the dspace support for pdf-thumbnails, and Tony helped fix the layout problems that popped up when the thumbnails came online. I also imported the collection metadata into the vufind index on devcat with our vufind-OAI harvest tool and this xsl.

We interviewed a couple good candidates for the systems department position. The poor candidates each endured three hours of meetings, and we interviewed two candidates in a day, so it was exhausting for everyone. Two more candidates to go next week.

I checked in a couple vufind patches. One patch adds a link to a record's holdings screen that auto-populates the AUBIExpress (illiad) request form. The other patch fixes the call-number refinement in the search facets that I broke a while ago replacing AJAX facets. I think Clint will probably get the ok to release the patches next week.

Finally, I was able to setup authentication with dspace credentials for the minnows data-submission webapp. This bLog post has more details.

Sunday, November 14, 2010

2010/11/8-11 - whoops!

This week had good news and bad news - bad news first.

There was a little excitement this week as we accidentally released a bug to our live vufind server. On Monday we flipped the switch on a VM Clint had prepared for release that included a bug (that I wrote - ugh!) in the AJAX handler that marries a call-number to its proper item in a search result. The bug had gone unnoticed on our test server for over a week, and didn't lead to serious problems till Monday night, when search results started popping up with wacky call numbers. The reference desk paged systems, but the page went unnoticed till Tuesday morning (before I came in), when Denise and Jon called Clint at home on vacation. Clint quickly found the bad commit I had checked in, and reverted the bug back out. I keep expecting someone to throw a fit, but people have been cool, so maybe I'm too cynical.

The good news was that Claudine signed off on the metadata massaging that my import program runs through to marry her collection of scans with catalog MARC records for import into d-space. This project has been lingering since June - hopefully the end is in sight. I'll import the rest of the collection on Monday, and setup some initial XSL for Marliese to index the collection into vufind on devcat.

Finally, the vufind bug inspired me to setup some vufind regressions. The following e-mail excerpt explains what I did.



--- Reuben Pasquini 11/13/10 8:52 PM ---
Hi Clint!

I setup a framework for vufind regression tests that we can
add to over time.
    http://code.google.com/p/littleware/source/browse/?repo=catalog#hg/VufindRegression/src/main/scala/edu/auburn/library/tool/vufindTest

You can download a build of the test runner at:
    http://devcat.lib.auburn.edu/tests/vufindTest.zip
- just run 'vufindTest.bat'.

The test suite currently just runs through 3 sets of tests against devcat.
The first test runs a "Harry Potter" search (
    using HtmlUnit http://htmlunit.sourceforge.net/
), and verifies that the results include a couple expected records,
and that the call-numbers are correct on those 2 records.

The second test loads a specific Harry Potter record, then
checks the title,
verifies that the record has some holdings information,
and checks that the "campus delivery" cookies are set.

Finally, the third test accesses
    http://devcat.lib.auburn.edu/vufind/AuburnTestSuite.php
which runs through a suite of PHP unit tests
    http://devcat.lib.auburn.edu/cgi-bin/hgwebdir.cgi/vufind/file/tip/web/AuburnTestSuite.php

Anyway, we can start running these regressions before releasing new code.
I can set you up with a copy of the code if you want to add some tests.
I want to add tests that directly access some of the AJAX services, 
add a test fixture for Solr.php,
and setup a Hudson server (http://hudson-ci.org/) to
automatically run the tests whenever we commit some code to the vufind repository,
but I probably won't get to that for a while ...

Let me know if you run into problems with the test runner ...

Cheers,
Reuben

Sunday, November 7, 2010

2010/11/1-4 - election day!

It was another quiet week at the library. I spent some time updating the littleware build configs to simplify setting up the build for a new project. The build.xml files for each project now share a master build-template that defines some common cross-project build rules, and automatically installs ivy. I also updated the ivy.xml files to define compile, client, server, ee_client, ee_server, and test standard configurations; and to reference latest.integration dependency versions where appropriate. I hope to setup a hudson continuous integration server in the next few months to automate the test, artifact release, and server deploy processes.

I put most of the code in place that integrates the Google spread sheet data with the ACES collection d-space import tool. I hope to have a version of the collection on the server next week.

Finally, I helped with a couple small vufind bugs. Clint worked hard this week to release a few patches to the production server. There's some kind of big "vufind, love it or hate it" meeting next week. There are plenty of flaws with our vufind setup, but it has a lot of advantages over our only alternative.

Tuesday, November 2, 2010

2010/10/25-28 - Happy Halloween

I got a few nice things working this week. First, the multi-step minnows submission process is running. I scheduled a meeting for Thursday, but Jon forgot about it, so we'll try again next week.

Next, I integrated Barbara Bishop's gifs of the library maps into our devcat vufind install. I used ImageMagic to down-rez Barbara's images, and convert them to PNG, and wired up vufind to popup a YUI-2 panel to display the map. Barbara and Marliese gave some good feedback, but we haven't released the code to catalog yet - bla.

When working on the maps thing I stumbled across the fact that our "retrieve X from stacks" request system consists of a patron writing down what he/she wants and handing it to circulation. I'd like to try to setup an online system - submit a request, and get an e-mail or text when the item is at circulation kind of thing.

Finally I implemented a MultiVoyager.php vufind driver for some people on the vufind e-mail list. I sent out the following e-mail, but haven't heard back from anyone. Bla.


--- Reuben Pasquini 10/31/2010 4:57 PM ---
Hi David,

I was able to get a simple 2-ILS system working at Auburn on our dev server.
Here's what I did.

*. Setup a solrmarc import that prepends an ILS-specific prefix to each record.
    You can get something working quickly by modifying the 'id' rule in
    the solrmarc marc.properties to:

      id = script(customId.bsh),getCustomId("PREFIX")

    where customId.bsh has:

import org.marc4j.marc.Record;
import org.marc4j.marc.ControlField;


/**
 * Generate a custom id as prefix + 001
 */
public String getCustomId( Record record, String prefix ) {
    return prefix + ((ControlField) record.getVariableField( "001" )).getData();
}

.........................................

*. I wrote a 
      MultiVoyager.php
driver that reads
     MultiVoyager.ini
and maintains a list of Voyager.php driver instances.
I attached a copy of MultiVoyager.ini, or
you can clone a copy out of our Mercurial repository:
     http://catalog.lib.auburn.edu/cgi-bin/hgwebdir.cgi/vufind/ 

The MultiVoyager.ini file maps a prefix to and ILS config -
an example follows.
The "EMPTY" keyword indicates no prefix.
This example works for records that look like
     12345 (no prefix)
and 
     AUM12345 (AUM prefix).

MultiVoyager.ini:

[Catalog]
configList = EMPTY,AUM
defaultDriver = EMPTY
[EMPTY]
host        = bla
port        = 1521
service     = VGER
user        = bla
password    = bla
database    = bla
pwebrecon   = http://pooh.lib.auburn.edu:7008/vwebv/holdingsInfo 
[AUM]
host        = bla
port        = 1521
service     = VGER
user        = bla
password    = bla
database    = bla
pwebrecon   = http://pooh.lib.auburn.edu:7008/vwebv/holdingsInfo 

........................................

*. I had to patch a few other php lines here and there too
    to get the ajax load of holding status working right,
    modify Voyager.php to accept a config in the constructor
    rather than load Voyager.ini, ... - stuff like that.
    The complete patch is here, but our vufind code is
    pretty far out of date compard to vufind.org:
       http://catalog.lib.auburn.edu/cgi-bin/hgwebdir.cgi/vufind/rev/9e94166d5e0b 


........................

Hope this code works for you - let me know how it goes.

Cheers,
Reuben


-----Original Message-----
From: Reuben Pasquini [mailto:rdp0004@auburn.edu] 
Sent: Monday, October 18, 2010 1:20 PM
To: Harmon, Kelly; vufind-general@lists.sourceforge.net; Osullivan L.
Subject: Re: [VuFind-General] Anyone merging more than one
VoyagerDatabaseintoVuFind?

At Auburn we currently only index one Voyager database, but
we also OAI harvest other collections.
An easy trick is to just prepend a unique prefix before the
normal bib-id.
For example - we have 'SLEDGE143' and '143' records:
   http://catalog.lib.auburn.edu/vufind/Record/SLEDGE143 
   http://catalog.lib.auburn.edu/vufind/Record/143 

I think you'll have to hack at least 2 things to get this scheme to
work:

  *. Modify Drivers/Voyager.php to manage connections
    to multiple Oracle servers, and choose a connection
    based on the id-prefix.

  *. Modify the solrmarc vufind.properties file, and replace
          id = 001, first
    with something like
          id = my001WithPrefixFunction
    ... something like that.

I'd like to do something like this myself, but it keeps falling off
the end of my TODO list.
Let me know if you need help implementing this, and I'll set aside a 
day or two to give it a try.

Cheers,
Reuben