2010 Final Report
NDNP Awardee Final Report
(July 1, 2008 – June 30, 2010)
NEH Award Number: PJ50038-08
Project Name: Washington State Digital Newspaper Project
Awardee Institution: Washington State Library
NDNP State: Washington
Project Director: Marlys Rudeen
Project Manager: Laura Robinson
Report Date: September 30, 2010
Project Activities
Project Administration
Upon notice of
the National Digital Newspaper Program (NDNP) award, an internal project team at
the Washington State Library (WSL) was created, a search started for a project manager, and a request for proposal submitted to select a digitization and conversion vendor. By November 2008 a project manager was hired, an advisory committee convened, space for
the project was arranged at
the University of Washington (UW) and WSL, a vendor was selected, and equipment was purchased.
Throughout
the grant period
the project director and project manager attended
the annual awardee meetings and turned in interim project and fiscal reports. A production schedule was created and batches delivered throughout
the project. After communication from
the Library of Congress (LC) regarding
the upload and availability of newspaper pages, news was announced via
the Washington State Library blog and press releases for each title were sent to media,
the advisory committee, museums, schools, and o
ther interested parties to announce
the availability of new content on Chronicling America.
Due to state budget reductions,
the project team lost two Digital Project Librarians (one at
the beginning of
the grant and one half way through), both of whom had a total of 13 hours a week devoted to
the NDNP grant. While this likely affected
the speed of QA and some coordination with CONSER catalogers, QA time was made up by help from o
ther WSL staff, and
the refining and automating of much of
the QA workflow with batch scripts.
The titles were extensively researched before and during digitization. A list of titles,
their lccns, and dates of coverage were delivered to NDNP staff at LC. Information about each title, including
the condition of
the film, where
the film was stored, which regions
the film covered and an estimated number of pages were compiled on
the project wiki: http://wiki.sos.wa.gov/ndnp.
WSL and UW worked with
their film service bureaus to check availability of master negatives and perform a technical analysis of
the film, testing density and resolution.
The content of
the film was also analyzed to ensure proper coverage of dates and minimize running into large and unexpected issue gaps.
A schedule for production was created around three major milestones of
the project (July 31, 2009; Dec. 31, 2009, and June 2010) estimating shipments 1 batch (18-20 reels) every 6-8 weeks.
The first sample reel was delivered to WSL and was tested and validated using
the DVV. On preliminary QA, some issues were found with
the sample reel but resolved in time to send
the sample reel to
the Library of Congress by February 1, 2009.
Some production delay was caused by
the adoption of new technical specifications for metadata and changes to
the Digital Viewer and Validator (DVV). With each change of
the DVV,
the scripts used to create
the metadata had to be changed and tested by our conversion vendor, Content Conversion Specialists (CCS).
Communication with
the scanning and text conversion vendors was handled via an MS Sharepoint portal hosted by Online Computer Library Center (OCLC),
the vendor hired to scan
the microfilm.
The project manager stayed in communication with
the vendors and participating partners through
the details of
the project such as testing, duplication, and scanning of film;
the delivery of
the images and conversion of
the images to text;
the delivery of batches and rework reports; and finally communication of accepted batches. Because of
the number of organizations involved and number of time zones and countries, communication and delivery often felt cumbersome.
A large part of project administration was a basic understanding of
the technical specifications of
the grant. This understanding was essential in making communication between
the partners possible as well as understanding why
there were delays in production. It was also much easier to create a more efficient workflow with a better understanding of
the validation process and
the technical specifications.
Title Selection
A large selection of titles were originally researched and vetted by a Selection Advisory Committee.
The preliminary list consisted of 94 titles researched and submitted as possibilities for scanning.
The committee reviewed
the list for intellectual content such as
the coverage of political, economic, cultural history, religious, and ethnic events from 1900-1910. O
ther considerations were geographic coverage and titles that served as
the “paper of record” for
their regions. Final ranking resulted in a final list of 23 titles.
The final list reflected titles of historical importance, geographically diverse coverage, and titles that had a complete (or nearly complete) run of issues filmed for
the time period selected.
In
the case that some film would have to be passed over because of bad film found during technical analysis, more titles were picked as substitutes. Four related titles; Ranch, Ranche and Range, Ranch (Seattle), and Washington Farmer; were added to
the final list that did not appear on
the preliminary list.
These titles were filmed as part of ano
ther grant after
the preliminary list was made.
They added important content about
the business of agriculture and life in
the Northwest during
the time period.
Microfilm Evaluation
While it was beneficial to first check
the positives for technical targets, reduction ratios, etc. it was not always possible to confirm
the quality of
the negatives this way. Since
the large majority of
the master negatives of
the selected reels were housed by service bureaus, technical analysis was conducted after our initial selection and during
the creation of
the second generation masters. During technical analysis, some film was deemed of lesser quality, but no titles were completely omitted from
the final list because of too high reduction ratios or bad resolution.
As a project partner, UW was charged with ordering its own film while WSL ordered
the creation and testing of its film through Proquest and
the Washington’s Office of
the Secretary of State’s Imaging and Preservation Division.
The master negatives of
the titles selected for review were tested to determine film density, resolution, and reduction ratio. This testing cost between $20-$40 per reel depending on
the vendor. WSL staff compiled any technical data within
their abilities (e.g. identifying reduction ratios from
the positive before title selection).
In 2009
the NDNP Technical Specifications changed making
the capture of much of
the reel technical analysis (e.g. density readings, resolution, etc.) optional. WSL determined that
the readings coming from different vendors on film that didn’t have targets were too subjective to be very helpful or meaningful. So much time was lost trying to get meaningful readings and educating vendors about
the specifications that we didn’t feel it was worth
the time and money involved in capturing data from non-targeted film.
The project manager, working with UW, hired a student from
the Information School and toge
ther
they evaluated
the microfilm and captured title, reel and issue data on a spreadsheet using Google Docs. Google Docs was chosen because of its use of versioning and
the easy access between locations and institutions.
The data was imported into an Access database and delivered to OCLC.
The data was also used for WSL’s QA processes which will be discussed in fur
ther detail in
the Quality Review section below.
The Google Docs spreadsheets were available to view online and were used to calculate and display reel page totals and project progress from
the project wiki.
Scanning and Text Conversion¶
A vendor was selected to do
the scanning and text conversion through a competitive Request for Proposal process that followed
the rules and guidelines of
the State of Washington. OCLC Preservation Service Center, which later became Back Stage Library Works (BSLW), was selected as
the vendor with
the most experience and proven record of working with microfilm, preservation quality scanning, and producing digital files to
the NDNP specifications.
Negatives duplicated by o
ther vendors were shipped to OCLC, after which
the film was scanned.
The images were
then shipped to CCS,
the sub-contractor hired by OCLC to process
the images, create
the OCR files in
the ALTO format, create
the derivative images and
the embedded metadata in each image file using
the DocWorks software, a workflow management product.
The batches were validated and shipped back to WSL where validation was verified and data and image quality were checked again. Error reports were shared with CCS and often data and images were ei
ther reworked and shipped to WSL via a second hard drive or delivered over ftp. When possible, WSL staff
then reintegrated
the reworked data and images into
the batches, re-validated
them, and shipped
them to LC.
Quality Review – After a general assessment of
the condition of
the film,
the images and data went through a three part screening process. Each newspaper page was first viewed as positive microfilm (a.k.a. service copies) by
the project assistant at UW. Much of
the reel and issue data was captured during this process.
The data was
then reviewed by
the project manager and delivered to OCLC/CCS. Each frame was again reviewed during scanning and creation of derivatives, while
the data was used to collate
the digital images and populate parts of
the xml and image headers. Once
the batches were delivered to WSL,
their validation was verified.
The images were again reviewed by staff at WSL (each JP2 and PDF is viewed in thumbnail form) and
the output data tested against
the input data by batch scripts written by
the project manager.
Delivery
At this point 100% of
the data and images have been delivered and accepted by LC. All essays have been researched, written, delivered to, and accepted by NEH staff.
Accomplishments
Washington’s participation in
the National Digital Newspaper Program accomplished goals and initiated fur
ther plans for future development of digital collections. 1) It produced files for over 100,000 pages of newspaper content that met
the NDNP standards and specifications for deposit with
the Library of Congress. 2) It initiated a move to create access to a state repository of digital newspaper content for Washington papers. 3) It has helped create a set of technical best practices to convert newspaper microfilm to digital formats that will serve as a model for future digital newspaper projects in
the state. 4) It has successfully demonstrated how an organized, collaborative statewide approach achieves more consistent, sustainable, and cost effective output of digital files for historical research.
The project has made available pages of newspapers around
the state from a significant period in Washington’s history. NDNP has also helped extend
the access and efforts of
the US Newspaper Project in Washington to microfilm and catalog newspaper collections. It created an Advisory Committee of representatives and experts to help achieve
the long term goals of
the project. So far
the Committee has accomplished an extensive survey of extant historical newspaper hard copy held by heritage organizations and libraries statewide as well as reviewed a preliminary list of Washington newspaper titles that met
the criteria for inclusion in NDNP.
Ano
ther development that has been initiated but remains ongoing is
the use of
the Washington State Digital Archive’s methodology for long term preservation of digital images and data. Staff has begun
the first steps of creating a digital archive of newspaper resources that will be maintained and housed by
the Digital Archives, with oversight and user interface development provided by WSL and agency web staff.
Audiences
The main audiences for this project are
the historical research professionals, students, teachers, and family researchers. While our goal was to cover major population centers of
the state we were also especially aware of regional researchers interested in areas of
the state where
the newspaper comprises one of
the only records of community life available. This is especially true in more rural populations of our state. During this grant period, we worked diligently to include papers from different geographical regions of
the State as well as papers that provided a recognized source for special audiences and interest groups. For example, papers that covered labor organizations, agriculture and farming, and
the first successful African American newspaper in Seattle were selected.
Evaluation
WSL has been happy with
the success of
the grant and
the learning and experience gained. Over 100,000 newspaper pages were digitized from microfilm and nearly all of those pages have gone live on Chronicling America. Researchers are able to view newspaper pages online for free that
they would have o
therwise had to travel to WSL or UW to research.
There has been a steady increase in page views in
the year our newspapers have been live. We’ve started to combine usage statistics to track of our local newspaper products (Pioneer Newspaper Collection) and
the newspaper titles for NDNP. With
the issuing of targeted press releases for each featured region, staff have seen some press coverage and received positive feedback from local community members and organizations, including a desire from several rural areas to have
their local papers digitized.
Evaluation of
the project workflow for
the past two years led to changes in
the work plan for
the second grant proposal. Since
the vendor used a sub-contractor, project staff was often trying to convey complex questions to, and transfer data between, staff in OCLC/BSLW’s office in Pennsylvania, CCS’ office in Germany, and
the actual work unit in Romania. While everyone did
their best, it was a frustrating and time consuming process that left project staff feeling that
they did not have sufficient control over
the schedule and workflow. It is anticipated that, even with
the steep learning curve
the software will require, bringing
the image processing and metadata creation in-house will address
these concerns.
An assessment of project strengths would include: 1)
the establishment of a strong, collaborative working relationship with
the University of Washington, 2)
the creation of an adaptable workflow for largescale newspaper digitization, and 3)
the start of a plan for implementing steps to carry out newspaper digitization beyond our NDNP grant.
Some assessed weaknesses have been 1) often underestimating timelines due to workflow or shipping constraints, 2) mishaps in communication between
the vendors and project staff that have caused delays, and 3) not utilizing local libraries and project partners as much as desired when planning outreach and promotion to local researchers and patrons. This will be a priority in
the second round of
the project.
Continuation and Long Term Impact¶
WSL has received ano
ther two-year NDNP grant to continue
the work started in 2008. Staff has also begun to plan and put infrastructure and equipment in place to integrate
the output of
the NDNP grants with existing digital newspaper content created at WSL. WSL has begun brainstorming ways to cover more areas of
the state, especially less populated areas that will likely not be covered in
the NDNP grant. Because of
the popularity of
the project and
the need to cover more areas of
the state, staff has planned to license
the workflow software docWorks and bring
the NDNP text conversion and derivative creation in house.
The hope is to build up staff expertise and expand newspaper digitization to o
ther areas of
the state and plan for continuation after NDNP.
This project has streng
thened our collaboration with
the University of Washington in preservation and access of
the state’s newspapers. It has also helped us initiate a move to digitize our assets in standard formats and prioritize
the use of standard data formats in seeing
the first hand benefits of sharing a research repository with o
ther states. We hope to fur
ther our collaborative partnerships in
the area of outreach and use of
the NDNP collections in
the next grant round.
Grant Products
Nearly all
the output created during
the grant period is available for public use and research. Products of
the NDNP grant include approximately 450,000 digital files (100,000 pages scanned as tifs, two forms of image derivatives per page, one xml OCR text derivative per page, and one xml METS file per issue and reel to package
the files), over 100,000 newspaper pages searchable online, essays for each title, updated bibliographic MARC records, press releases for each title, a project wiki that includes information and links to research about newspapers, and scripts and workflow processes to make post digitization QA more efficient.
Washington 2008-2010 Title List
Title City Dates Reels Pages
Cayton's Weekly Seattle, WA 1917-1921 1 738
Colfax Gazette Colfax, WA 1900-1912 8 4814
Colville Examiner Colville, WA 1907-1922 5 6422
Commonwealth Everett, WA 1911-1914 1 322
Industrial Freedom Edison, WA 1898-1901 1 250
Leavenworth Echo Leavenworth, WA 1904-1922 6 5596
Pullman Herald Pullman, WA 1888-1922 11 13650
Ranch Yakima, WA 1894-1912 8 10090
San Juan Islander Friday Harbor, WA 1894-1914 11 6818
Seattle Republican Seattle, WA 1896-1902 5 4668
Seattle Star Seattle, WA 1899-1910 22 24325
Tacoma Times Tacoma, WA 1894-1922 24 25023
Washington Socialist Everett, WA 1914-1915 1 601
Yakima Herald Yakima, WA 1893-1912 15 8539
Totals 119 111856O
ther products include
the following presentations, wiki, posts and articles:
- Blog posts: http://blogs.secstate.wa.gov/library/?tag=NDNP
- Wiki: http://wiki.secstate.wa.gov/ndnp/
- Presentations:
- Aug 2010 - Washington and Pacific Northwest Library Association, Victoria, BC
- Aug 2010 - NDNP Annual Meeting, Washington, DC
- Nov 2009 - Museum Computer Network, Portland, OR
- Oct 2009 - Washington Library Media Association, Yakima, WA
- Oct 2009 - LITA National Forum, Salt Lake City, UT
- Aug 2009 - NDNP Annual Meeting, Washington, DC
- April 2009 - Washington Library Association, Spokane, WA
- Mar 2009 - Washington State Council on Social Studies, Chelan, WA
- Article: Mar. 2010 - Microform & Imaging Review article: The Evolution of Newspaper Digitization at the Washington State Library
Template
COVER PAGE
Provide
the following information in
the order requested:
- type of report (interim or final performance report),
- grant number,
- title of project,
- name of project director(s),
- name of grantee institution (if applicable),
- date report is submitted.
NARRATIVE DESCRIPTION
The items listed are provided as guidance to
the project director in developing
the narrative description of project activities. Because projects vary considerably, not all items will be relevant to a particular project. Please feel free to organize this portion of
the report in
the way that most clearly presents what has taken place during
the grant period.
Final Performance Report
Using
the project description and plan of work that were approved by NEH as a point of departure,
the final performance report should address
the following subjects:
1. Project Activities
- Provide a description of the major activities that occurred during the grant period.
- Indicate the reasons for omissions and changes in project activities.
- If project performance was affected by changes in key project personnel, explain why the changes were made and how performance was affected.
- When federal matching funds were a component of the award, summarize fund-raising experiences and the major factors believed to be responsible for success or failure in raising third-party support.
- For projects involving computer applications, describe any changes that were made in the method of data entry, the specific data to be encoded, software, hardware, file systems, or search strategies.
- Briefly describe any efforts that were made to publicize the results of the program.
2. Accomplishments
- Compare the accomplishments of the project in quantitative and qualitative terms with the objectives proposed in the application.
- When project goals were not achieved, indicate what plans there are to complete the project after the grant period, how project activities will be funded, and when they are likely to be completed.
3. Audiences
- Describe the audiences for the project. Indicate the nature, size, geographic reach, sex and age of the audience and assess the impact that the project had on this audience. What kinds of new or previously underserved audiences did the project attract? It is particularly important to compile quantitative information for this section of the report. Please include data on all screenings and broadcasts, if applicable.
- How much of an increase in visitor flow or membership did your organization experience as a result of the project?
- In the case of grants whose purpose was to affect a number of other institutions, include in the report a complete list of participants and appropriate statistical profiles that show the impact of the project by geographical region (if possible), kind of institution, and level and type of participant.
4. Evaluation
- Was an evaluation of the project performed? If so, briefly describe how the evaluation was performed and by whom.
- Describe the results of the evaluation and your own assessment of the program. Discuss both the weaknesses and the strengths of the program. A discussion that includes how problems were dealt with will be more helpful to NEH staff than one that focuses exclusively on the project's successes.
- How did the public respond to the project? What did they like or not like? What anecdotes, statistical summaries, feedback from web sites, viewer remarks, or examples of media coverage can you provide that would help to assess the project's success?
5. Continuation of
the Project
- Indicate if there are any plans to continue the project after the grant period because of the success of the program and the interest it has generated.
- When there was a commitment on the part of the grantee institution to continue a program after the grant period, explain how the commitment will be honored. If the program will not be continued, provide a detailed explanation for the change in plans.
- What kinds of new collaborative partnerships were formed (or strengthened) between your institution and other organizations (e.g., museums, historical societies, schools, universities, community groups, special interest groups, etc.) as a result of the project? Will these new partnerships continue and, if so, how?
6. Long Term Impact
- What kinds of long-term impact (such as spin-off programs, use in the classroom or other indicators of continuing interest) will result from the project? * How did the project affect your institution's ability to attract additional non-federal financial support, either for the project or for activities that grew out of the project?
- What effect did the project have on the public's perception of your institution and on your plans for future projects?
7. Grant Products
- Indicate what grant products were produced during the course of the project and any future publication or distribution plans for materials resulting from grant activities.
Normally,
the information that is to be included in a final narrative description can adequately be covered in a report that does not exceed ten typewritten pages.
APPENDICES
Enclose with
the report one copy of any supporting material that would contribute to an understanding of
the project and its accomplishments to date. This would include:
- representative samples of completed work,
- preliminary products such as conference or workshop papers,
- course syllabi and manuals,
- written evaluations of a project,
- consultant reports, if required,
- articles submitted to journals,
- illustrated field reports,
- copies of published announcements or other formal efforts to recruit participating scholars,
- copies of any mailing, fliers, newspaper releases or articles, or other media coverage.
It is not necessary to append work in progress, such as draft chapters of a book or o
ther manuscript materials. However, unless o
therwise specified in
the conditions of
the grant award, two copies of any publication, film, videotape, or slide presentation resulting from
the grant should be forwarded to
the Endowment with
the final report.
TopResources
Top