OONI Highlights: 2017-2019
This post was originally published on the OTF website.
Back in 2012, the Open Observatory of Network Interference (OONI) was born with OTF support
to create the first open methodologies that would enable anyone around
the world to measure internet censorship and other forms of network
Today, OONI comprises a global community that runs its censorship
measurement software (OONI Probe) in
more than 200 countries every month.
Millions of measurements have been collected from tens of thousands of
networks in 236 countries since 2012, all of which are openly published
on OONI Explorer – arguably the largest
publicly available resource on internet censorship to date.
Most of this growth is thanks to OTF support
over the years.
In this post, we share highlights from what the OONI team accomplished
between 2017 and 2019 with OTF Internet Freedom Fund support.
Image: Comparison of OONI Probe mobile 1.0 with OONI Probe mobile 2.0
(both released between 2017-2019)
To enable the internet freedom community to measure censorship events
around the world, we first needed to:
Make OONI Probe available on more
platforms (such as Android and Windows);
Improve OONI Probe usability;
Make OONI Probe more useful to the internet freedom community
(by including new features based on community requests);
Improve and expand upon OONI’s methodologies to collect richer
measurements and detect more forms of internet censorship.
To this end, we released:
New OONI Probe mobile app for Android and iOS. For the first
time, it became possible to run OONI censorship measurement tests
with the tap of a button!
Revamped version of the OONI Probe mobile app.
OONI Probe mobile 2.0 includes many new features, improved design
New OONI Probe desktop app for Windows and macOS. This
desktop app has a similar design and UX as the revamped OONI Probe
New OONI Probe Command Line Interface (CLI).
This is for Linux users who may want to run OONI Probe on servers
and contribute more stable measurements.
New OONI Probe measurement engine. Previously, the
OONI Probe mobile app was powered by the Measurement Kit library.
To enable the integration of tests and iterate more quickly with higher confidence in code quality, we re-wrote the OONI Probe
measurement engine from C++ to golang and released the OONI Probe engine.
New OONI Probe tests. These include a new Telegram test (and
rewrite in Go), OONI
implementations of M-Lab’s NDT
and DASH performance tests,
speed test), a DASH rewrite using HTTPS, as well
as follow-up measurements for DNS over TLS fallback, DNS over HTTPS fallback, and
SNI blocking detection.
Jafar: a censorship simulation tool. To ensure that our methods
work as expected and to make OONI Probe more resilient, we built a
censorship simulation tool to
run experiments and improve the robustness of our tools.
Methodology for investigating internet blackouts. We published a
which outlines our proposed methodology for investigating internet
blackouts from the edge of the network. We also published a blog post which
explains how to examine internet blackouts through existing public
Expanded global coverage of censorship events
Image: Monthly OONI measurement coverage worldwide
Every month, millions of OONI measurements are collected from
thousands of networks in more than 200 countries, shedding light on
censorship events around the world.
This expanded coverage is most likely due to the improvement of OONI’s
censorship measurement tools. The following OONI measurement stats show
a large uptick in February 2017, following the initial launch of the OONI Probe mobile app (which
made censorship measurement much easier).
Image: OONI monthly measurement statistics (previously hosted on the
Expanded measurement coverage means that more OONI Probe tests are run in more countries and
on more networks, enabling the internet freedom community to detect many
more cases of internet censorship around the world.
Between 2017 to 2019, we collaborated with our local partners (and other
community members) on the publication of the following 32 research
reports covering censorship events around the world:
Timely evidence of global censorship events
Image: OONI Explorer (revamped in 2019)
Now, OONI measurements from around the world are openly published in
near real-time. This means that as soon as anyone runs OONI Probe anywhere around the world, their test
results are (automatically) openly published on OONI Explorer within minutes (unless if they
have opted-out from publication). It is therefore now possible to track
internet censorship events around the world in near real-time, thanks to
the creation of a new fast-path pipeline.
Data analysts and researchers may be interested in the OONI API (launched in 2017) for downloading the raw
measurements in JSON format to perform their own analysis.
Image: OONI API
Given that other tools (such as OONI Explorer) depend on the OONI API, the use
of the API should be limited to light-weight queries (to avoid affecting
the performance of tools that rely on it).
To support batch consumption of OONI data, we have made a full copy of
the OONI PostgreSQL MetaDB available. This enables any third party
to setup a read-only replica of the OONI MetaDB, allowing them to run
heavy database queries on their own infrastructure for batch analysis.
This is the same database that also powers the OONI API and OONI Explorer. To enable adoption, we published
that explains how to set-up a copy of the OONI MetaDB.
Thanks to the Amazon Open Data program, the
whole OONI dataset can be fetched from the ooni-data Amazon S3 bucket, and we published a blog post explaining how to do
Rapid response to emergent censorship events
Image: OONI Run
To promote rapid response to emergent censorship events, we created a
new platform, called OONI Run, to enable
the coordination of customized OONI Probe testing.
Through OONI Run, you can generate a link based on specific websites that you are interested in
testing, and share that link with OONI Probe users so that they can test those
websites. This has been useful in various cases for measuring emergent
censorship events around the world.
Last year, for example, IPYS Venezuela
and Venezuela Inteligente coordinated an
OONI Probe measurement campaign
amid protests and political turbulence, sharing OONI Run links with Venezuelans to test YouTube,
Instagram, and Twitter across the country. This enabled the rapid
collection of relevant measurements, supporting the publication of a
providing evidence of social media blocking in Venezuela. Quite
similarly, OONI Run links were used to
coordinate testing in Egypt to collect evidence on the blocking of bbc.com and alhurra.com
As social media, news websites, and circumvention tools often get
blocked during political events around the world, we embedded several OONI Run buttons on the OONI website to enable the testing of
Image: OONI Run buttons embedded on OONI website
These sorts of buttons can be generated through the OONI Run platform.
All of the above requires a community: humans who are interested in
defending a free and open internet, and willing to run censorship
measurement tests from local vantage points (despite potential risks).
To empower censorship research participation in the internet freedom
community, we established 17 new partnerships (between 2017 to 2019)
with the following organizations:
We started establishing partnerships with digital rights organizations in
mid-2016, and the new partnerships listed above expand our network to 27
partnerships overall. Through these partnerships, we aim to support
local efforts in increasing transparency of internet censorship. To this
end, we collaborate on OONI network measurement and provide OONI data
Between 2017 to 2019, we collaborated with our partners (and other
community members) on updating 20 Citizen Lab test lists for the
Democratic Republic of Congo (DRC),
Nigeria, South Sudan,
To support local OONI community engagement efforts, we created and
published several resources, such as OONI workshop slides,
a Frequently Asked Questions (FAQ)
section, and an OONI Glossary. We
also started an OONI Community Interview series
and created a short documentary on
internet censorship in Cuba.
Thanks to OTF Community Lab support, we had the
opportunity to host the OONI Partner Gathering in 2017: a two-day
event that brought all of our partners together to share skills and
knowledge on censorship measurement research. We subsequently held
several other community gatherings, such as our monthly online meetings
and in-person meetups at international conferences and events (such as
RightsCon and the Internet Freedom Festival).
The OONI team is immensely grateful to the OTF for supporting the growth
of the OONI-verse. We look forward to continuing to support the internet
freedom community in the fight against internet censorship.