INTERIM PERFORMANCE REPORT NARRATIVE

NDNP Awardee Interim Performance Report
NEH Award Number: PJ-50038-08
NDNP State: Washington
Submitted By: Laura Robinson, Project Coordinator
Report Date: July 31, 2009

1. Changes in Workplan or Methodology

QA - The first batches were received and reviewed. A workflow for dealing with data or image errors has been established but continues to be refined. Most data errors in the batches are fixed in-house while images are sent back to the vendor via ftp to be reprocessed. Some common errors to look for during QA and other hints and tips were provided by Deb Thomas during her site visit in May. The data and digital image QA process has been created and tested. Using the detailed documentation of the NDNP METS data structure provided by LOC, we created scripts to verify that the evaluation data (input data) matches the metsrecord xml files (output data). The pdfs are checked by staff. For now, every pdf is copied to a temporary location while staff conducts review.

Documentation - A project wiki has been established and is used to collate project documentation, title selection, and ongoing research. Links to evaluation spreadsheets (Google Docs) are also available on the wiki. An effort to keep information on the internet helps to keep the project mobile and aid in collaboration.

Communication - OCLC hosts a SharePoint portal used for data delivery and communication. Semi weekly phone calls happen with OCLC and monthly phone calls are scheduled with all sub-contractors. Communication and coordination with multiple organizations in multiple countries has proven to be a slow process.

Infrastructure - We've worked to find funding to increase server capacity for the batches to live locally. Further work will be required to manage and preserve the local copy of files long term. An intermediary server is being set up to provide remote access to pdfs during QA.

2. Selection or Film Acquisition/Evaluation Activities

The final selection list was delivered to LOC in February, 2009 (included below). In late June, the decision was made to substitute the Bremerton News with the Walla Walla Evening Statesman due to a lack of complete microfilm and local funding to finish microfilming originals.

3. Award Activities with Collaborating Institutions

The University of Washington houses half the project by hosting the student work station and a work space for the project coordinator three days per week. The University also orders approximately half the film and manages the selection committee.

Other organization will aid in promoting the collection once images are loaded into Chronicling America.

4. Conversion and/or Vendor Selection

The first batch of approx 16,000 newspaper pages were delivered in June. Approx 20 errors were found and fixed. The vendor is having internal software issues and is only able to deliver a partial second batch of approx 6,000 - this batch is due to arrive in early July. We may not have the full 25% of images delivered to LOC in time to meet our July 31st deadline but the rest should follow soon after.

We're pleased with the quality of images and OCR data received from OCLC. We'd like to move to using the 2009 NDNP specs but have had to hold off due to sub-contractor's software. This has hindered the project somewhat.

5. Do you have any questions or comments regarding the use of the tools provided by LC (Digital Viewer Validation)?

We feel more confident with the DVV. There was a small hiccup with our first batch in that the subcontractor validated our first batch with the latest version of the DVV software and didn't tell us. After consulting with Ray Murray we learned that the file signatures are not upward compatible so it took us a while to figure out why our data wasn't validating once we received the batch.

6. Provide an update on your planned schedule for delivery of digital assets for the award, including samples, to the NDNP repository.

We should have approx. 22,000 images delivered by the end of July with approx. 10,000 to follow a few weeks after that. We are currently on track with our other milestones.

7. State briefly the status of any other digitized newspaper activities in your or collaborating partners’ institutions (non-NDNP-funded).

Since Jan. 1, 2009, the State Library has put up three more newspaper titles in their internal digital newspaper project. They are the Yakima Herald (1891-1892), the Vancouver Register (1865-1869), and the Puget Sound Herald (1861-1864), with a total of 1,822 page images. These newspapers are indexed to the article level by volunteers who are supervised by staff.

8. Please provide copies of or URLs for any public presentations or publications on topics related to your participation in NDNP (or describe future plans for such).

Presentation at WLA (Washington Library Association) in April related to using maps to display collections of digital newspapers (including NDNP) - slides

Washington State Library blog posts related to NDNP - posts

Planned poster presentation at LITA National Forum - display uses of various tools and techniques to make Washington's participation in NDNP a collaborative and mobile project.

Planned co-presentation at WLMA (Washington Library Media Association) - will talk about digital collections available to social studies teachers, including Chronicling America.

9. Describe any follow-up issues or questions you would like convey to the NDNP program committee as you begin your digital conversion project.

Please submit to your NEH Program Officer ([email protected]) with a copy to [email protected], by January 31, 2009.

NDNP Awardee Interim Performance Report
NEH Award Number: PJ-50038-08
NDNP State: Washington
Submitted By: Marlys Rudeen, Project Director
Report Date: January 31, 2009

INTERIM PERFORMANCE REPORT NARRATIVE

1. Changes in Workplan or Methodology

There have been few changes in the preliminary workplan to this point in the project. However, with the advent of the new DVV and adoption of the new technical specifications, there will be some adjustments in future months. At present, despite occasional bureaucratic delays, the project is on schedule for the first deliverables at the end of January.

The Project Coordinator, Laura Robinson, was hired and began work full-time in September. It was decided that it would be advantageous to have her work part of each week at the University of Washington (UW) campus. This enabled her to supervise student workers directly, and continue to work closely on selection issues with UW staff.

2. Selection or Film Acquisition/Evaluation Activities

Title selection is continuing, and the decision was made to focus on a particular time period with a wide geographic distribution. Accordingly the selection committee looked for titles that covered 1900-1910, with at least one title for each region and the significant towns of the period: Bellingham, Everett, Seattle, Spokane, Tacoma, Vancouver, Walla Walla, Wenatchee and Yakima. At a meeting between UW and WSL on Sept. 12, 2008 both organizations decided on the first titles to be duplicated/purchased. Since both UW and WSL own master negatives, it was decided that each organization would be responsible for managing its own negatives and obtaining the required duplicates for the titles they own.

Both partners have spent some time working with their service bureaus to get acceptable 2nd generation negatives and the required density readings. Since many of WSL’s negatives are held by a third party, this required some research to confirm the location and availability of certain titles.

While this was happening, Laura hired a work study student and began developing the workflow for examining the film and collecting metadata. She began by experimenting with an Access database supplied by our vendor (OCLC) and with Excel spreadsheets. However it was not always possible to get the required programs on the computer available for the student worker, so they began to use the spreadsheet function on Google Docs, a free web-based set of applications. This has worked well, ensuring that Laura and the student can reach the files from any location and providing the ability to link the film evaluation data to the NDNP wiki (see section 3). They are consulting with OCLC to firm up the respective metadata responsibilities.

The first reel was scanned and run through OCLC’s OCR process, and the data and images were shipped to WSL. Laura ran it through the DVV to make sure it validated before beginning the quality assurance process. Using the sample reel, the internal quality assurance processes are being developed.

We did find some incorrect metadata and are creating a work plan to best deal with such errors. One such example of an error was the incorrect attribution of ownership of the tiff images to ‘Minnesota State Historical Society’ in the validated data files (_1.xml files). Laura investigated and determined it came from embedded metadata in tiff files. She located a command-line tool used to extract, write, and view embedded data in image files and is working with the vendor to develop a plan of correction.

Since the latest version of the DVV has arrived, the decision was made to adopt the 2009 technical specifications for metadata on future reels.

3. Award Activities with Collaborating Institutions

A contract with the UW was executed to allow the Washington State Library (WSL) to reimburse UW for two functions: purchase of duplicate film from negative masters owned by UW; and payment of work study worker(s) at the UW Library. Arrangements were also made to provide office space for Laura and student worker(s) at UW.

Laura has established a wiki for the project to use for maintaining files and communicating. The NDNP wiki is located at http://wiki.secstate.wa.gov/ndnp/MainPage.ashx

The wiki provides a method of storing and making available a variety of documents generated during the project, and will provide a coherent record of the project. News and updates also circulate on a project listserv to all participants and committee members.

4. Conversion and/or Vendor Selection The RFP for scanning and OCR was issued on Sept. 15, 2008 and responses were due Oct. 15, 2008. Three proposals were received and evaluated by staff according to a procedure and scoring process developed in consultation with WSL’s contracts office. On Oct. 24, 2008, WSL announced that the successful vendor was OCLC. WSL and its contract office immediately began working on the contract which is now completed.

While contract negotiations were going on, the portable hard drive and mailing cases were purchased, named (after boats in the Washington State Ferry fleet), and shipped to the vendor in Bethlehem, PA.

Film examination and metadata collection were completed for the first title, five reels of the Seattle Republican and related titles. The first reel will be used as the sample reel to be submitted to LOC by January 31, 2009.

A kickoff meeting with OCLC staff has been scheduled for January 7, 2009 to discuss metadata specifics, project structure, communication, and scheduling of work and shipments.

5. Do you have any questions or comments regarding the use of the tools provided by LC (Digital Viewer Validation)?

The DVV has been very helpful and generally easy to use. There were very minor issues with the install and a suggestion made to LOC staff to add text to the documentation for future users. We’ve found that the GUI has been less helpful and more limiting than the command-line options of validation. We intend to use the GUI and perhaps Adobe Bridge for visual validation of the image files.

We are hoping to create scripts and that can output the batch, reel, and issue metadata into various formats for testing and validation.

6. Provide an update on your planned schedule for delivery of digital assets for the award, including samples, to the NDNP repository.

The hard drives purchased for the project have a capacity of 1 TB, requiring between 18-20 reels to fill. Shipments of drives will take place every 6-8 weeks. The table below represents the list of titles and possible substitutes divided by six month periods to assist the staff members responsible for acquiring the negatives, performing the film evaluation and quality control.



7. State briefly the status of any other digitized newspaper activities in your or collaborating partners’ institutions (non-NDNP-funded).

Since July the State Library has put up two more newspaper titles in their internal digital newspaper project, the Yakima Herald (1889-1890), and the Puget Sound Herald from Steilacoom (Mar. 1858-1860). Several more titles are in process. This internal project has been significantly slowed down by budget and staff cuts, and by the fact that some of the personnel involved have significant roles in the NDNP project.

8. Please provide copies of or URLs for any public presentations or publications on topics related to your participation in NDNP (or describe future plans for such).

WSL’s NDNP related news:

• http://blogs.secstate.wa.gov/digitization/index.php/category/digital-collections/newspapers/ndnp/
• Mar. 13, 2009 Washington State Council on Social Studies, Chelan, WA. Presentation on digital collections and primary resources will highlight Chronicling America and NDNP.
  

9. Describe any follow-up issues or questions you would like convey to the NDNP program committee as you begin your digital conversion project.

It is our understanding that the new DVV includes validation of embedded metadata in the image files. Because of such issues as described above we are looking forward to having more robust checking of the metadata in the image files.

Please submit to your NEH Program Officer ([email protected]) with a copy to [email protected], by January 31, 2009.