User Guide: OONI Test Lists Editor

Last updated: 7th September 2022

Test Lists Editor

The OONI Test Lists Editor enables the public to review and contribute to the lists of websites (“test lists”) that are tested for censorship by OONI Probe users around the world.

This user guide provides step-by-step instructions on how to use the Test Lists Editor to review and contribute websites for censorship testing.

Upon reading this guide, we hope you will feel empowered to contribute to test lists!

The following table of contents should help with navigating the guide.

About the Test Lists Editor

What are test lists?

Test lists are lists of websites that are tested for censorship by OONI Probe and tools developed by other projects, such as Censored Planet.

Since 2014, these lists have been publicly hosted on GitHub by the Citizen Lab with the goal of encouraging community review and contributions. As a result, these lists have been dynamically updated on an ongoing basis over the years.

There are 2 categories of test lists:

Test lists include a wide range of different types of websites based on 30 standardized categories (such as news media, political criticism, and human rights content). Since these lists are tested by OONI Probe users on local networks (who may experience bandwidth constraints), they usually only include up to 1,000 URLs.

While test lists usually include some websites that are known to be blocked, many sites are not censored locally when they are added to test lists. With test lists, we aim to discover website censorship (by identifying the blocking of sites that were previously accessible), not only confirm it.

We encourage you to learn more by reading our documentation about test lists.

What is the Test Lists Editor?

The Test Lists Editor is a web platform through which you can review and contribute to lists of websites tested for censorship. This platform includes all of the Citizen Lab test lists, and users can add URLs, edit existing entries, and propose the removal of entries through a web interface (without using GitHub).

User contributions (submitted through the Test Lists Editor) automatically end up as pull requests on GitHub, which are peer-reviewed by the Citizen Lab, OONI, Censored Planet, and Netalitica. Once these pull requests are reviewed and merged, the contributions are integrated into test lists and (automatically) prioritized for OONI Probe testing.

Why contribute to test lists?

Discovering cases of website blocking (such as the current blocking of bbc.com in Russia) really depends on which websites you test.

For example, if a specific human rights website is blocked in a country, but it’s not included in a relevant test list, it won’t get tested by tools like OONI Probe, which means that relevant test results will not be openly published.

You can play an important role in ensuring that your country’s test list includes websites that are worth monitoring for censorship.

Updating test lists requires local knowledge, an understanding of which websites are relevant, commonly accessed, and more likely to be blocked in light of a country’s social and political environment.

Websites constantly change (e.g. domains expire, domains change, new websites are created), and what is sensitive to blocking changes over time. It is therefore important that test lists are reviewed and updated regularly.

The OONI Probe results of website testing are automatically published as open data in real-time.

Using the Test Lists Editor

You can use the Test Lists Editor through the following steps.

Step 1. Access https://test-lists.ooni.org/

Test Lists Editor

Step 2. Add your email address in the Email slot (we do not store email addresses).

Test Lists Editor

Step 3. Click Login. This will send a link to your email address.

Test Lists Editor

Step 4. Click the link (“Please login here”) sent to your email to log into the Test Lists Editor.

Test Lists Editor

Step 5. Once logged into the platform, select a test list through the drop-down menu.

Test Lists Editor

In this example, we have selected the test list for Azerbaijan.

Test Lists Editor

As part of reviewing a test list, you can:

Adding websites

To add a website to a test list:

Step 1. Add the URL (e.g. https://news.az/) to the URL slot.

Test Lists Editor

Important: Please ensure that the URL is typed correctly. If it is mis-typed, OONI Probe will not test the intended website and this will potentially lead to inaccurate test results.

A few things to keep in mind:

To ensure that a URL is typed accurately, please check how it appears when accessed from a normal browser.

Step 2. Select a category (e.g. News Media) through the Category drop-down menu to categorize the URL (e.g. https://news.az/) you have added.

Test Lists Editor

You will see the 30 categories that are used for categorizing websites included in test lists. These categories range from news media, culture, and human rights issues to more provocative or objectionable categories (like pornography or hate speech). Please choose the category for each website you are adding.

Optional. If you would like to add relevant context pertaining to the added URL, please add it under Notes. The Notes section can be useful for describing the type of URL added, particularly since the standardized categories are quite broad. As an example, you can write “Site of the political opposition, reported to be blocked during 2016 elections” to provide context that may be useful to researchers.

Test Lists Editor

Step 3. Click Add.

Test Lists Editor

Your addition will be saved, and you can continue adding many more websites.

Step 4. When you are done making changes, click Submit to propose your changes.

Test Lists Editor

Your changes will automatically appear on GitHub as a Contribution from test-lists.ooni.org pull request.

Test Lists Editor

We may provide feedback through the comments section of your GitHub pull request. In this case, you can answer directly in the comments section, or send us an email to contact@openobservatory.org referencing your GitHub contribution.

Test Lists Editor

Once we have integrated your contributions to the relevant test list (by merging your GitHub pull request), your recently added URLs will automatically get prioritized for OONI Probe testing.

Test Lists Editor

Editing test list entries

You can edit existing test list entries to:

To edit existing test list entries:

Step 1. Click the edit icon in the row of the entry.

Test Lists Editor

This will make the editing screen appear.

Test Lists Editor

Step 2. Edit the entry by updating the URL, updating the category, or adding useful context in the Notes.

Test Lists Editor

Step 3. To enable third-party review of your changes, please briefly explain why you are making these changes in the Comment section.

Step 4. Click Done when you’ve completed your edits.

Your proposed edits (along with any other changes) will appear at the start of the Test Lists Editor.

Test Lists Editor

Step 5. When you are done making changes, click Submit to propose your changes.

Your changes will automatically appear on GitHub as a Contribution from test-lists.ooni.org pull request. Please check the GitHub page to see if reviewers have shared feedback.

Test Lists Editor

You can respond to reviewer feedback directly in the comments section of your GitHub pull request, or by sending us an email to contact@openobservatory.org referencing your GitHub contribution.

Deleting test list entries

You can propose the deletion of a test list entry if a website is no longer available (e.g expired domain), or if the website is not relevant to the specific country.

To delete test list entries:

Step 1. Click the delete icon in the row of the entry.

Test Lists Editor

This will make the delete screen appear.

Test Lists Editor

Step 2. Please explain why you would like to propose the deletion of a URL (e.g. “Website no longer exists”).

Step 3. Click Delete to propose the deletion of that URL.

Your proposed deletion (along with any other changes) will appear at the start of the Test Lists Editor.

Test Lists Editor

Step 4. When you are done making changes, click Submit to propose your changes.

Your changes will automatically appear on GitHub as a Contribution from test-lists.ooni.org pull request. Please check the GitHub page to see if reviewers have shared feedback.

Test Lists Editor

You can respond to reviewer feedback directly in the comments section of your GitHub pull request, or by sending us an email to contact@openobservatory.org referencing your GitHub contribution.

If you have any questions, please don’t hesitate to reach out.

Thanks for contributing!