Year in Review: OONI in 2019
Maria Xynou
2020-03-20
As the new decade begins, we publish this post to share some OONI
highlights from 2019. We also share some thoughts for 2020.
New Apps
Revamped OONI Probe Mobile App
Image: Comparison of old and new OONI Probe mobile app
We launched OONI Probe mobile 2.0! OONI Probe
is free and open source software
designed to measure internet censorship
and various forms of network interference.
The new OONI Probe mobile app has been revamped entirely based on
community feedback and features requests from the last years, following
the initial launch of the OONI Probe mobile app in February 2017.
Highlights
from the revamped OONI Probe mobile app include:
Major UI overhaul. The new app has an entirely new design and
UX, simplified navigation, and improved presentation of
test results.
Enhanced website testing. You can test the websites of your
choice directly through the OONI Probe mobile app! You can also
test the URLs included in the Citizen Lab’s country-specific test lists,
and choose which categories of sites to test.
Overview of test results. You can now have a bird’s eye view of
all your OONI Probe test results, enabling comparison.
Data usage. You can keep track of how much of your mobile data
the OONI Probe app is using.
Try out the new OONI Probe mobile app! It’s available for
Android,
F-Droid,
and iOS.
Thanks to support from the Localization Lab community, the app has been
translated to
the following 15 languages: Arabic, Farsi, Chinese, Russian, Hindi,
Spanish, French, Portuguese, Catalan, Turkish, Italian, Albanian, Greek,
Slovak, German.
New OONI Probe Desktop App
Image: New OONI Probe desktop app
Throughout 2019, we worked on developing the first native OONI Probe desktop app, which
we plan to publicly launch soon!
Supporting OONI Probe on Windows has been a popular community request
over the last years. We previously supported macOS and Linux users with
a command line version of OONI Probe that could also be run from a web user interface. However, setting
up OONI Probe from the terminal presented a technical barrier for many
community members. Furthermore, many human rights defenders globally
(that are interested in running OONI Probe) use Windows, which wasn’t
previously supported.
Now, Windows and macOS users will be able to easily install and run OONI
Probe through the upcoming new desktop app. We have generally
tried to create a seamless experience between the OONI Probe mobile and
desktop apps, which is why you will notice that the design and UX of the
two apps is very similar, and that they include many of the same tests.
However, the new OONI Probe desktop app includes some different
features in comparison to the OONI Probe mobile app:
Circumvention tool tests. You can measure the blocking of
Tor and
Psiphon with the new OONI Probe
desktop app. These tests will be integrated into the OONI Probe
mobile app as well within 2020.
Website testing. Due to bandwidth constraints on mobile, the
default setting is configured to test a random sample of URLs
within 90 seconds. In the OONI Probe desktop app, on the other
hand, you will test all URLs included in the Citizen Lab’s
country-specific
and global test lists
in one run. You have the option to limit the number of websites
you test through the OONI Probe desktop app settings.
While the new OONI Probe desktop app was released in 2019, we plan to
publicly launch it soon (once it’s more polished). So stay tuned!
Near real-time publication of OONI measurements
As of November 2019, OONI measurements from around the world are
published in near real-time! This means that as soon as anyone runs
OONI Probe anywhere around the world,
their test results are (automatically) openly published within minutes (unless they have
opted-out from publication).
This is a major milestone because for the first time, the public can
track global censorship events in near real-time. And this is thanks
to our colleague Federico Ceratto, who joined the OONI team in May 2019
and built a new fast-path pipeline that makes
instant analysis and publication of measurements from around the world
possible.
Revamped OONI Explorer
Image: Next Generation OONI Explorer
We launched OONI Explorer 2.0!
Consisting of millions of network measurements collected from thousands of networks
in more than 200 countries since 2012, OONI Explorer is arguably the largest publicly
available measurement resource on internet censorship to date.
Highlights from the next generation OONI Explorer
include:
Revamped country pages. Through the new OONI Explorer country pages, you can learn which
websites were most recently blocked (if block pages are
served), how many measurements have been collected per OONI Probe test over time, and how that
compares across networks in each country.
Search tool. With the new OONI Explorer search tool, you can filter
measurements to view which websites are confirmed blocked (i.e.
block pages
are served) and which websites presented network anomalies (and
therefore may be blocked). You can also filter measurements per
ASN to compare censorship across networks.
Measurement pages. Through the newly designed OONI Explorer measurement pages,
you can gain an overview of test results, as well as access to the
raw measurement data, which you can download in JSON format.
Censorship highlights. The landing page of the new OONI Explorer
shares some examples of interesting cases
that you can uncover through OONI data. These include
politically-motivated censorship events, media censorship, the
blocking of LGBTQI sites, and censorship changes around the world.
OONI PostgreSQL MetaDB
To support batch consumption of OONI data, we have made a full copy of
the OONI PostgreSQL MetaDB available.
This is the same database that also powers the OONI API and OONI Explorer.
The OONI PostgreSQL MetaDB enables you to set-up a read-only replica of
the OONI MetaDB, allowing you to run heavy database queries on your own
infrastructure for batch analysis. To enable adoption, we published
documentation
that explains how to set-up a copy of the OONI MetaDB.
Improved measurements
New OONI Probe tests
We expanded our methodologies by developing the following new OONI Probe
tests:
For follow-up measurements, we wrote the following tests:
To ensure that our methods work as expected and to make OONI Probe more
resilient, we built a censorship simulation tool (called Jafar) to run experiments and improve
the robustness of our tools.
New OONI Probe measurement engine
We released a new OONI Probe engine!
Over the last years, the OONI Probe mobile app was powered by the C++
Measurement Kit
library. To enable the integration of tests, improve its
maintainability, and iterate more quickly with higher confidence in code
quality, we re-wrote the OONI Probe measurement engine from C++ to Go.
The new OONI Probe engine
powers both the OONI Probe mobile and desktop apps. We plan to complete
the process of making the apps rely entirely on the new golang engine in
2020.
New partnerships
Our network of partners grew in 2019!
Throughout the year, we had the opportunity to establish 6 new
partnerships with the following digital rights organizations:
With our growing coalition of partnerships, we will continue to measure
and report on internet censorship around the world.
Research publications
In collaboration with our partners (and other community members), we
published the following 13 research reports covering censorship
events around the world:
OONI workshops and presentations
As part of our community engagement and outreach activities in 2019, we
facilitated workshops and presented OONI at the following conferences
and events around the world:
FOSDEM 2019 (Belgium), Session: “Designing for Activism: An overview of the UX overhaul of OONI Probe” (2nd February 2019)
BarCamp Chiang Mai (Thailand), Session: “OONI workshop” (16th February 2019)
Vietnam Cyber Dialogue (Spain), Session: “OONI censorship measurement”
(31st March 2019)
Internet Freedom Festival 2019 (Spain), Sessions: “OONI
community feedback: Improving censorship measurement research“,
“Conducting network measurements in censorship hotspots – lessons
from the field“, “Multi-disciplinary teams, journalistic
capabilities, and methodological protocols to monitor and
investigate internet censorship“, “OONI Partners Meetup”, OONI
presentation at the IFF Expo & Tool Showcase (1st – 5th
April 2019)
Digital Rights & Inclusion Forum 2019 (Nigeria), Session: “Uncovering Censorship Evidence”
(23rd April 2019)
Media for Democracy Forum (USA), Panel: “New and Emerging Threats to Media Freedom: What’s
Next?” (7th May 2019)
RightsCon 2019 (Tunisia), Sessions: “Supporting the KeepItOn campaign with network measurement data”,
“Measuring internet shutdowns“,
“Detecting, understanding, and countering censorship of Wikipedia“,
“Methods: Measuring freedom of internet”,
“OONI Partners Meetup”, “OONI workshop (Day 0)“ (11th-14th June 2019)
Citizen Lab Summer Institute 2019 (Canada), Sessions:
“Censorship Measurement Campaigns: A decentralized approach to measuring and responding to internet censorship
events” (1st August 2019)
Chaos Communication Camp 2019 (Germany), Sessions: “OONI: Measuring Internet Censorship Around the World”,
“OONI Hackathon” (22nd-25th August 2019)
OONI partners and other community members facilitated OONI workshops
around the world too! Examples include the Open Culture Foundation’s OONI workshop in Taiwan and
an OONI workshop hosted in Venezuela
for local journalists.
To support OONI community engagement efforts around the world, we
created and published several resources. These include:
OONI workshop slides. Interested in facilitating an OONI
workshop? Download the OONI workshop slides
and adapt them to meet the needs of your communities.
OONI Frequently Asked Questions (FAQ). The new OONI FAQ addresses many questions that
community members ask about OONI, OONI Probe, OONI Explorer, and
OONI data.
OONI Glossary. The new OONI Glossary provides brief
explanations for terms commonly used in the OONI-verse (within
OONI apps, documentation, and research reports), as well as for a
number of technical terms used in the internet freedom community.
OONI Run buttons page. Is social media reportedly blocked in a
country? To support rapid response to emergent censorship events,
we integrated thematic OONI Run buttons on our website to
enable the testing of specific types of websites.
OONI-verse
Thanks to the OONI community, many measurements were collected from far
and wide.
Every month in 2019, more than 8 million OONI Probe measurements were
collected from more than 4,000 networks
in more than 205 countries around the world.
Image: Monthly OONI measurement coverage worldwide
Based on this global measurement coverage, many cases of internet
censorship were detected and reported
around the world. Apart from our reports, many other cases were reported
by community members who made use of OONI data as part of their own, independent
research efforts.
Some highlights include:
Blocking of thousands of websites in Egypt. Through the use of
OONI Probe and OONI data, Egypt’s AFTE published a research report
on the collateral blocking of thousands of websites in Egypt.
Media filtering in Russia and Crimea amid presidential elections. Through the use of OONI data, OTF
Information Control Fellows Ksenia Ermoshina and Igor Valentovitch
published a research report
that investigated, contrasted, and compared internet censorship in
Russia and Crimea during the 2018 Russian presidential elections.
Measuring internet censorship across Ukraine. The Digital Security Lab of Ukraine
created its own “local OONI”, coordinating OONI Probe censorship
measurement efforts across Ukraine. Based on their analysis of
OONI data, they published several research reports documenting the
blocking of websites
and how that varies across regions and networks in Ukraine.
Online petition site blocked in Cuba. Through the use of OONI
Probe and OONI data, a Cuban non-profit organization, called
YucaByte, published a
report
on the blocking of change.org in Cuba.
Freedom on the Net 2019 reports. Many Freedom House reports on Freedom on the Net
cited OONI data, including those for Venezuela,
Zimbabwe,
Zambia,
Nigeria,
Kenya,
Jordan,
Iran,
Indonesia,
Ethiopia,
Egypt,
and Cuba.
Overall, OONI data became more actionable in 2019, as community members
used OONI Probe and OONI data in many interesting ways. Below we share a
few highlights from 2019:
OONI data supported a High Court petition in Pakistan. Back in
November 2017, we reported
on the blocking of several media websites, Facebook (facebook.com
and Facebook Messenger), YouTube, Twitter, Instagram, WhatsApp,
and Telegram in Pakistan amid protests. Our report – which shared
OONI network measurement data collected from Pakistan – was cited
as evidence in a petition filed in the Islamabad High Court by
Pakistan’s Media Matters for Democracy, challenging the blocking of
media websites and popular social media platforms during
the protests. The petition – filed through
Charahgar –
challenged
the Pakistan Telecommunication Authority’s (PTA) authority to
block websites without due process. As an outcome, the Islamabad
High Court declared
that the Pakistan Telecommunications Authority’s interpretation of
Section 37 of PECA is against the Constitution, and that they
cannot regulate content without following the due process, the
principles of transparency, and oversight. This is a landmark
victory for internet freedom in Pakistan.
Venezuelan partners nominated for 2019 Gabo Award. Through the
use of OONI Probe and OONI data, IPYS Venezuela (along with Venezuela Inteligente /
VEsinFiltro) has been measuring
internet censorship in Venezuela since 2014. They have created a
network of volunteer OONI Probe users across Venezuela, who
coordinate on measuring and reporting on emergent
censorship events. Through the use of OONI data, they have
published numerous research reports, documenting internet
censorship events in Venezuela on an ongoing basis. Their
Intercortados project was
nominated
for the 2019 Gabo Award,
among the 10 best innovation projects.
Harvard’s AccessCheck project powered by OONI data. Harvard’s
Berkman Klein Center for Internet and Society launched
AccessCheck, a
platform that enables you to check the accessibility of websites
around the world. We are excited that AccessCheck is powered by OONI data, along
with data collected from virtual private network (VPN) endpoints
and virtual private servers (VPS).
We are excited to see how community members make use of OONI data in
2020! If you’re interested in working with OONI data and need support,
please don’t hesitate to reach out.
2020
There’s no doubt that 2020 will be a challenging year for everyone
around the world.
In light of the escalating global impact of COVID-19 (coronavirus),
many digital rights conferences and events have already been cancelled
(such as the Internet Freedom Festival
and
RightsCon),
and we’ll likely see less of each other in person this year.
Nonetheless, we look forward to connecting more with you online.
Join us on the OONI Slack channel to chat
with the OONI team and community members from around the world. Please
feel encouraged to use the OONI Slack channel as a social space amid
social distancing:
a space to connect with others, share ideas and thoughts, and to support
each other.
We facilitate monthly community meetings on the OONI Slack channel to receive news and updates from the
community, collect community feedback on OONI tools and methodologies,
address questions, and foster discussions on internet censorship issues.
We usually facilitate these meetings at 14:00 UTC on the last Tuesday of
the month, but we confirm these details on the ooni-talk mailing list.
We will also explore opportunities to facilitate online workshops and
events throughout 2020. If you’re interested in facilitating an online
OONI workshop to engage people with censorship measurement, we encourage
you to do so and we share OONI workshop slides
that you can download and adapt.
Ensuring a free and open internet during an emergency, like the current
pandemic, is more important than ever. But a few days ago, state-owned
CANTV blocked access to a coronavirus information portal in Venezuela.
Measuring internet censorship can therefore be a question of public
safety.
Throughout 2020, we will continue to improve upon the OONI Probe apps and methodologies. We will continue to
improve upon OONI Explorer and our data
analysis capabilities. We will continue to report on censorship events around the world, and to serve the
internet freedom community.
We hope you all stay safe
during these difficult times.