2010/05/29 - somebody reads this bLog!

I just sent the following e-mail to the vufind-general list. It turns out somebody actually reads this bLog - awesome!

Hello!

I didn't know anyone actually read my bLog!
Anyway - we run d-space at Auburn for ETD
     http://etd.auburn.edu
and we're working on a more general repository server too.
We have a vufind server that's still beta:
     http://catalog.lib.auburn.edu
We'll hopefully work out some final issues with
our index this summer.

We have a vufind-import tool that builds on top
of SolrMarc to manage MARC record imports
   (Voyager Oracle database -> SolrMARC -> SOLR),
but also supports ( XML -> XSL -> SOLR )
and (OAI -> XSL -> SOLR) pipelines as of a few
weeks ago.
   http://lib.auburn.edu/AuCataloging/jar/vygr2vfnd.jnlp
I'll eventually update the apps on the Google code site too -
     http://code.google.com/p/littleware/
I'm still working on a few patches.
We hope to eventually have a set of automatic cron-jobs
in place that run our import tool to nightly
OAI-harvest collections we want to index in vufind.

Anyway - the tools XML and OAI import tabs rely upon you to 
write your own XSL that converts your XML import records
into SOLR <doc></doc> blocks that conform to your vufind index's
schema and include data that the Vufind display engine can handle.
The bLog post includes the XSL file we applied at Auburn
to harvest a collection from Georgia Tech's repository:
     http://au-reuben.blogspot.com/2010/05/20100512-14-oai-harvest-to-vufind.html
One note - be sure to configure your XSL so that your import assigns
unique ids to each collection.
For example - at Auburn we use numbers (1,2,3,...) for our
Voyager records (from the 001), 
then have different prefixes for our Content-DM collections -
so the "Carline Dean" collection from Content-DM as a 'CF' prefix:
(CF1, CF2, ...
   ex: http://catalog.lib.auburn.edu/vufind/Record/CF377
).

The XSL import process for us at Auburn may be simpler than for
other vufind installs.
We forked our vufind codebase off vufind.org last year:
   http://catalog.lib.auburn.edu/cgi-bin/hgwebdir.cgi
, and customized our vufind engine to remove dependencies on MARC
that existed at the time.
Every once in a while we go back and merge in code from vufind.org
(like the great mobile them
        http://catalog.lib.auburn.edu/mobile/
), or at least steal ideas (disable AJAX facet loading).
I understand the vufind RC2 code now supports a "Driver" framework
that allows vufind to deal with non-MARC data in the index,
but we haven't merged that code in, and I don't know if anyone 
takes advantage of that or not.  I think most vufind installs just convert
XML metadata to MARC then run the MARC through SolrMarc to
get non-MARC collections into the index.  I can't stand MARC.

Anyway - feel free to give our import tool a try.
The two big challenges you'll need to work through are
figuring out the XSL, and hacking your vufind install's PHP or driver logic
to deal with the non-MARC data that the tool puts into your index.
Of course you'll also need to enable d-space's OAI-harvest webapp
if you're not already running that on your d-space repository.

Good luck!

Cheers,
Reuben


>>> Eoghan Ó Carragáin <eoghan.ocarragain@gmail.com> 05/28/10 1:48 PM >>>
Hi Paolo,
I can't think of anyone using DSpace & VuFind, but I may be wrong. From
memory, Colarado Statue University use Vufind to expose their Digital
Repository (digitool from Exlibris), Auburn University use it with OCLC's
ContentDM repository (we do this at the National Library of Ireland too, but
it isn't live yet), & Greg Pendelbury in USQ is working with Vufind &
Fedora/Fascinator. Reuben frorm Auburn University recently posted a blog
about his work with harvesting metadata into Vufind using OAI-PMH:
http://au-reuben.blogspot.com/2010/05/20100512-14-oai-harvest-to-vufind.html

Are there any specific areas you need help with?

All the best,
Eoghan

On 28 May 2010 16:20, Baglioni, Paolo <Paolo.Baglioni@eui.eu> wrote:

>  Hi All,
>
> Has anybody configured VuFind to search their DSpace repositories? If so,
> any hints on how to go about this?
>
> Many Thanks
>
> Paolo
>
> =========================
>
> Paolo Baglioni
>
> Library Systems Analyst
>
> European University Institute
>
> =========================
>
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
>
>
> _______________________________________________
> VuFind-General mailing list
> VuFind-General@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/vufind-general
>
>

2010/05/24-27 - Honor's College and Tuskegee

It was a good week in cataloging at the library. Midge and I met with Kathie Mattox from the Honors College on Tuesday. The meeting went very well. We worked out the following procedures to manage the online Honors College thesis collection.

• Student procedure
  1. Check thesis draft with Kathie at HC. Kathie gives the student a copy of the library disclaimer required to eventually publish the thesis to the library online server.
  2. Student finishes thesis, gets signatures on the various forms, makes three copies, and delivers the copies and forms back to Kathie at HC.
  3. Student posts a pdf-version of the final thesis to the library server.
• For Kathie
  1. At some point before a student can submit his/her thesis pdf to the server, Kathie must enable that student's account on the server by filling out a web form. Reuben will give Kathy some training on how to do this, and write up instructions.
  2. Kathie will receive e-mail when a student submits a thesis to the server. Kathie will login to the server and review the student's thesis online, then either "Accept" or "Reject" the student's submission via a web form. Reuben will give some training and write up some instructions.

Midge finished her final draft of the disclaimer that we'll ask the student to sign when submitting a thesis to the collection. The Dean wants Auburn's lawyers to sign off on the disclaimer - hopefully that won't turn into a mess.

I went to Tuskegee on Thursday to help with a few setup tasks for their new DSpace repository server. We managed to get a few things done; we registered the Glassfish v3 server as a windows service, fixed some issues with new user creation and thumbnails, and scheduled an automatic nightly backup. Dana and Rod are doing a great job administering the server, and adding collections. They plan to enable public access to the server in the next month or so.

We also tackled a couple other tasks this week. Liza is helping work through some issues with the new click-tracker code I released last week, and Clint is using the new vufind import tool to test some final changes to vufind's Solr index.

2010/05/19-20 - Glenn to retire

The big news from Thursday's monthly staff meeting is that Glenn will retire in August. Glenn is a great guy - hope he has fun.

Otherwise I've just been multitasking between a few things. Jon and I rolled out the AuCataloging update yesterday. Everything seems to be working ok. I'm going to try to flip the click-counter data-view over to a PrimeFaces datatable, and do away with the existing custom YUI-table code that now throws errors in Chrome and Safari.

Clint and I met to merge in his Voyager-to-Vufind tool patch that fixes a record-format handling bug. I also just finished adding a log file to track bib ids that fail to transfer into VuFind - usually because Oracle goes down for reboot in the middle of the night.

Finally I exchanged a few e-mails for the Repository project. We're going to try to meet with Kathy from the Honor's College next week. Amy in the Business Office also indicated that we can include Auburn's annual reports in a collection, but she hasn't replied yet to a follow up e-mail I sent. I'll also touch base with Boyd next week on the LADC senior projects.

2010/05/17-18 - vufind patch and redlib1 build

I spent most of Monday and Tuesday coding the vufind patch described below, and building out the redlib1 server upgrade for the /AuCataloging webapp. I'll try to meet with Jon tomorrow to rollout the upgrade to the library web server.

--- "Reuben Pasquini" 5/18/2010 12:22 PM ---
Hi Clint,

I just updated the main vufind server
     http://catalog.lib.auburn.edu/ 
with this patch
    
http://catalog.lib.auburn.edu/cgi-bin/hgwebdir.cgi/vufind/rev/5f9d22ec998a 
to load the facet and search data in the
same index search rather than load
the facets via AJAX.
Let me know if you spot any problems.

Cheers,
Reuben

2010/05/12-13 - OAI harvest to Vufind

The vufind-import tool now supports simple OAI harvest. I finally got an OAI-harvest to VuFind tool working. As a test I was able to harvest most of the 15000 records in Georgia Tech's ETD collection into Vufind on devcat.

• Georgia Tech ETD: http://smartech.gatech.edu/handle/1853/4760
• OAI Harvest URL: http://smartech.gatech.edu/oai/request?verb=ListRecords&set=hdl_1853_4760&metadataPrefix=oai_dc
• Devcat Vufind: http://devcat.lib.auburn.edu/vufind/Search/Home?lookfor=hdl.handle.net&type=all&submit=Find&filter[]=building:%22Auburn%20University%20Digital%20Library%22

The tool harvests the OAI metadata, then applies an XSL transform, and finally posts the result to the Solr server: http://devcat.lib.auburn.edu:8080/solr/biblio

The XSL for the GTech ETD collection is below. You can see what the import-app looks like here: http://redlib1.lib.auburn.edu:8080/AuCataloging/jar/vygr2vfnd.jnlp If you click on the 'OAI' tab, and reset the XSL-file field to a copy of the XSL below, then you can test the OAI import.

I spent some time yesterday setting up the Honors College Collection in our repository server. Boyd told me yesterday that the LADC staff that we need to talk with about setting up an online collection for LADC senior projects are not available during intercession. He'll catch up with them when summer term starts, and hopefully things will work out.

gaTech.xsl:
--------------------------------

<?xml version="1.0" encoding="UTF-8" ?>


<xsl:stylesheet version="1.0"
     xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
     xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" 
     xmlns:dc="http://purl.org/dc/elements/1.1/"
     >
  <xsl:output method="xml" indent="yes"/>
<!--
http://smartech.gatech.edu/oai/request?verb=ListRecords&set=hdl_1853_4760&metadataPrefix=oai_dc 

-->

  <!-- pass-through rule -->
  <xsl:template match="@*|node()">
      <!--
     <xsl:copy>
       <xsl:apply-templates select="@*|node()"/>
     </xsl:copy>
        -->
  </xsl:template>


  <xsl:template match="/oai_dc:dc">
   <doc><xsl:apply-templates select="*" />
       <field name="format">Electronic</field>
       <field name="collection">Auburn University ETD</field>
       <field name="building">Auburn University Digital
Library</field>
       <field name="publisher">Georgia Institute of Technology</field>
       <field name="allfields">
   <xsl:for-each select="*">
       <xsl:value-of select="." /><xsl:text>
</xsl:text>
   </xsl:for-each>
      </field>
     </doc>
  </xsl:template>

  <xsl:template match="dc:identifier[1]">
    <field name="id">GTechETD<xsl:value-of
select='substring-after(.,"http://hdl.handle.net/1853/")'/></field>
    <field name="url"><xsl:value-of select="." /></field>
  </xsl:template>


  <xsl:template match="dc:creator">
     <field name="author"><xsl:value-of select="." /></field>
  </xsl:template>

  <xsl:template match="dc:subject">
    <field name="topic"><xsl:value-of select="." /></field>
    <field name="fulltopic"><xsl:value-of select="." /></field>
   </xsl:template>


  <xsl:template match="dc:title[1]">
     <field name="title"><xsl:value-of select="." /></field>
  </xsl:template>

  <xsl:template match="dc:description">
    <field name="description">
      <xsl:value-of select="." /></field>
  </xsl:template>

</xsl:stylesheet>

2010/05/10-11 - working away

I've just been dividing my time today and yesterday between configuring the new AuCataloging server on redlib1, configuring the new Honors College collection on repo, and adding XML+XSLT and OAI-harvest support to the vufind-import tool. Everything is going ok, but everything takes more time than I'd like.

The big new is Lori is back from her teaching internship. She seems none the worse for ware.

2010/05/06 - Honors College Theses Collection

I walked over to the Honors College today, and had a nice discussion with Kathie Mattox about establishing an online collection for the Honors College senior theses. The Honors College is very happy to support the collection if the library sets it up. I'll try to get our new repository in shape to support the Honors College collection over the next few weeks.

I'll try to walk over to LADC next week. Boyd is doing some leg work over there, but I haven't heard back from anyone yet.

2010/05/03-05 - Tuskegee and stuff

I spent Tuesday helping Dana and Rod at the Tuskegee University archive setup a DSpace repository server. By the end of the day we had a basic server up and running for them to experiment with. I told Dana that I'd be willing to come back out in a month to help configure the server with a Tuskegee look, enable thumbnails, and configure the collection metadata. Hopefully they like d-space - we'll see how it goes.

I spent some time this week on a spread sheet and diagram that consider library operations from a macroscopic view. I had a couple lively discussions after I sent an ill-informed e-mail protesting a staffing change, and that got me thinking about how things work at the library. Right now I have a model that organizes library services into three broad categories: manage research material, learning commons, and digital and special collections. The services in each category primarily serve different communities - graduate and faculty, undergraduates, and non-faculty/non-student respectively. I sent a description of the model to a few people - we'll see what they think.

Library what if ...

Suppose we group MDRL, e-journal and database management (ez-proxy, link resolver, ...), web site and server admin (ETD, Voyager, Vufind), and transfer all that responsibility and budget to an "online resources" subgroup under OIT. OIT gets almost all the materials budget, the systems department, and a group of 4 or 5 e-journal and database specialists (Jack, Paula, ...).

What's left for the library ? A staff budget over $4 million to manage a book budget under $250K, reference, and ILL.