This week we dig into the procedures that power the NRS Web Continuity Service. We are a multi-faceted service, dealing with numerous stakeholders and subject areas. With that in mind, we need to ensure our processes are efficient and effective, to help us deliver a high quality web archive.
But what do we mean by ‘high quality web archive’? In web archiving, quality can be related to three elements:
Completeness – how much of captured website’s links, text, downloads etc. the crawler has been able to access and capture
Behaviour – how much of the navigational functionalities within the captured website snapshot have been preserved, compared to the live site
Welcome to our blog! Over the course of few weeks, we will take Open Book readers on a tour of NRS’s new Web Continuity Service. Web archiving and Web Continuity represent an exciting new era for archiving at NRS, providing a digital tool that directly supports our mission to,
“collect, preserve and produce information about Scotland’s people and history, and make it available to inform present and future generations.”
Stay tuned for bite-sized articles on how this new service operates, and how it will contribute to the development of Scotland’s national archive collection and support the Scottish Government’s transparency agenda.
Websites as archival public records and the ‘looking glass’ into government
Nowadays, when a member of the public wants to understand something about government, the first source they will likely check is an official government website (probably found via Google).
In this multi-channel era, government websites have a critical role to disseminate official, trusted information, so that the government remains accountable and transparent to the citizen.
As a result, government websites form an integral part of the public record. National archives, who capture, preserve and make available public records, are therefore taking steps to capture a representative record of this modern aspect of government. To do so, national archives are creating web archives. Web archives have been around for some time. Nevertheless, the process of web archiving is technically challenging: more on that in our next blog post.
If done well, web archiving has the potential to dramatically alter the way we record, preserve, and analyse the activities of our government and wider society.
Selecting and capturing government websites, evidencing how these change over time, and making the output of this archiving process clear, reusable and interoperable, can create a powerful ‘looking glass’ into modern official business. It can also do this in a scalable and consistent manner.
Furthermore, emerging research is indicating that web archives may form the single most important contextual record for understanding society in the last twenty years, and will continue to do so. Here’s some examples to ponder:
Do you want to understand how US institutions reacted to the September 11th attacks? The Library of Congress has a web archive collection dedicated to this which is free to access, including this archived snapshot from the US Department of State, dated 12th September 2001.
Turning closer to home, the snapshots of the Scottish Parliament website, now captured by National Records of Scotland, provide a navigable resource for the business of Scotland’s devolved legislature, with time-stamped captured content available on Current Bills, MSPs, and special events and visitors to Holyrood.
Observant readers will quickly notice some unusual features about these archived pages; they all have arresting headers to show the user the page is archived and when this occurred, and some of the original dynamic functionality such as search, unfortunately may not work.
What is key though is that these archives have attempted to capture information from these websites as completely and accurately as possible.
In the next blog, we will explore the core technology behind web archiving, its technical challenges, and how archives (and NRS) are responding to this new era of collecting.
For many archivists embarking on a Digital Preservation programme, the biggest challenge is knowing where to start. NRS has been helping local authority archivists to overcome this initial hurdle with two new tools to help archivists get started with digital preservation.
Last summer, two Heritage Lottery-funded Skills for the Future trainees, seconded on year-long placements, joined our the Digital Records Unit to help local authority archivists get started with digital preservation. A year later, they have developed two tools which will enable archivists to take their first steps in digital preservation. These are the ‘Digital Preservation Guidance for Local Authorities’ and the ‘Capacity Planning Tool – Counting The Bits’.
The Digital Preservation Guidance for Local Authorities is written guidance tailored for local authority archivists and other staff who will be responsible for digital preservation. This may include records managers, information managers and IT staff. The guidance is written in simple, non-technical language so that the audience can be as wide as possible. The aim is to give anyone working directly or indirectly with digital preservation a clear idea of what it means and what is involved, to facilitate the start of the process.
The Capacity Planning Tool helps local authority archivists to calculate what their digital storage needs are. It does this by helping them to estimate how many digital records currently sit within their organisation, and helping them to estimate what percentage of records they need to preserve in the long term. Although this might sound like a simple task, it can be surprisingly difficult for archivists to make these calculations on their own. It is not uncommon for archivists to have no relationship with the digital records being created within their organisation. Even finding out where digital records are kept can be a challenge!
The Tool works by asking archivists complete a simple series of questions about records currently held within the organisation. It allows the archivist to identify and prioritise the departments that are most important to them. It also provides guidance about how to go about sourcing the necessary information. Once this information has been input, the tool makes some calculations based on factors such as estimated percentage of total records to be captured in the digital strategy; number of records that sit on the live system; and number of records that will be created to sit within the repository. These calculations equip users with tangible figures that can be used to initiate a conversation about resourcing needs. The results are shown as easy to use graphs and tables, which archivists can use to demonstrate their capacity requirements in an easy to understand format.
Like many projects that appear complex at the outset, the key to a successful digital preservation strategy is to break down the process into its constituent parts. Together the Capacity Planning Tool and the Guidance act as a foundation upon which a successful strategy can be built.
Our Digital Records Unit is launching two new digital preservation tools this summer. These guidance and capacity planning tools have been specifically developed for Scottish local authorities. They are the product of a 12 month project and will assist local authority archivists and record managers get started with digital preservation.
The guidance tool will help local authorities to understand and implement the steps needed to ensure that digital records are captured and preserved within the archive, while the capacity tool enables users to calculate their digital storage needs.
The events are aimed towards those currently working within Scottish local authorities, however other interested parties are also very welcome to attend.
The tools will be launched in Glasgow City Chambers on July 10th (book here) and in Aberdeen Town Hall on August 8th (book here).
Tickets are selling fast so be sure to register soon if you would like to attend, and spread the word to anyone who might be interested.
You can follow the events on Twitter, using the hashtag #scotladp and we’ll be livetweeting from @natrecordsscot.
We look forward to seeing you in Glasgow or Aberdeen!
