Dev Grant Spotlight — Webrecorder
This is the latest post in our series highlighting the builders and innovators creating new tools and services on the Filecoin network. Learn more about Filecoin Foundation’s Dev Grants program.
The Webrecorder project builds and maintains an ecosystem of open-source Web archiving tools that can be used by anyone to create high-fidelity and accurate archives of any web content. Webrecorder’s mission is to support ‘web archiving for all’, building browser-based tools to enable institutions all over the world to create web archives, from a single to running larger crawls in the cloud. Additionally, the company provides services to digital preservation specialists, activists, journalists, digital humanists and independent researchers and archivists to create their own web archives and store them in portable, open data formats that will be accessible in the future.
Webrecorder received a developer grant from Filecoin Foundation to improve areas of its open-source web archiving ecosystem while focusing on growth opportunities for the company through team expansion. Webrecorder’s goal is to improve current tools that support both cloud-based, larger-scale automated web archiving activities, primarily employed by libraries and archives around the world, as well as bring web archiving directly into the browser for anyone to use, through continued extension development and direct integration.
The company is also developing a portable format for storing web archives on both decentralized and centralized storage, which allows interactive archived Web pages to be loaded as a custom media format, like videos, 3D models or PDFs. This format can serve as a bridge for storing Web2 content on Web3 systems. Its portable data format, Web Archive Crawl Zipped (WACZ) and the associated data model is designed to make it easy to load Web archives of all sizes from IPFS and Filecoin.
“Webrecorder is the only organization that focuses on building open-source web archiving tools to enable the creation, storage and access to archives in a decentralized way. The focus is on tools, rather than building and maintaining an institutional archive,” said Webrecorder founder and lead developer, Ilya Kreymer “We want to empower anyone interested in digital preservation of web content and provide them with simple tools to create, archive and access important web content. This Filecoin Foundation grant helps us to ensure that our ecosystem of tools provides a way to ensure decentralized long-term storage of Web archive data of all sizes.”
Webrecorder will also explore approaches to improve trust in web archives, including a way to ensure data authenticity through cryptographic signing and verification at the web archive format level.
In addition, Webrecorder actively helps a number of web archiving initiatives throughout the world, including projects with the group called Saving Ukrainian Cultural Heritage Online (SUCHO), which works to prevent information and cultural memory from disappearing. The SUCHO group consists of hundreds of volunteer archivists, programmers and librarians using Browsertrix Crawler, through the Browsertrix.Cloud system, an open-source high-fidelity crawling system with a friendly user interface, as well as the ArchiveWeb.page Express system for fully browser-based Web archiving to record and archive Websites before servers lose connections or are destroyed.
“Our collaboration with Webrecorder is key to the IPFS and Filecoin mission: making a web that works for the most impacted users in critical situations, and ensuring the safety of the digital human experience for future generations,” said Dietrich Ayala, IPFS Ecosystem Lead at Protocol Labs. “Webrecorder provides the specs, libraries, tools and services to build bridges between the HTTP web and any of these new technologies, and bring the last 30+ years of the web along too.”
Webrecorder is looking for additional developers to help build and improve its open-source tools. Individuals interested to learn more about fully remote openings are to visit the Webrecorder Jobs page. Read more about the grant announcement.
The Filecoin Dev Grants program helps advance projects and aims to award additional grants to boost development in the Filecoin ecosystem. Check out our other grant recipients and learn how you can apply for Open Grants and Microgrants.
