Lepidopter Installation: Help Guides and Resources

Lepidopter guide

This guide explains step-by-step how to run OONI tests from a Raspberry Pi.

The code of the image as well as the build script can be found in the git repository.

Some compact documentation on how to install, build, copy to an SD card and run Lepidopter in QEMU is described here.

About Lepidopter

Lepidopter is an OONI Probe distribution for Raspberry Pis.

This enables users to contribute to the collection of network measurements consistently across time, without having to manually run OONI Probe from their computers and regardless of their technical skills. This distribution is configured in such a way that, when inserted into a Raspberry Pi and connected to a network, it will by default run OONI Probe once a day and send collected measurements directly to OONI’s measurement collector. It is also designed to automatically get updated to the newest version of OONI Probe.

All measurements collected by Lepidopter are published on OONI Explorer.

By using Lepidopter, you will be running the following OONI software tests via your Raspberry Pi:

The main advantages of collecting measurements through Raspberry Pis include the following:

Requirements

The following include a list of requirements to get started:

If your laptop does not have an SD card slot, please ensure that you have an SD card reader.

Downloading and verifying the Lepidopter Raspberry Pi image

Download the Lepidopter image

Consider downloading the xz compressed image for a significantly reduced file size compared to the zip archive. The signature to the xz compressed image can be found here. Note that this will require an extra program to be installed depending on your OS.

The default username and password of the image is: lepidopter

Verify Lepidopter Raspberry Pi image

Import the Lepidopter signing key

From a keyserver:

gpg --keyserver pool.sks-keyservers.net --recv-keys 625511968E240F24F6CFD0B6BA56AC5A53E9C7A4

Lepidopter team’s fingerprint (image signing key) is published here:

pub   4096R/0xBA56AC5A53E9C7A4 2016-05-22
      Key fingerprint = 6255 1196 8E24 0F24 F6CF  D0B6 BA56 AC5A 53E9 C7A4
uid   Lepidopter Team (signing key)

Set the trust level of the Lepidopter signing key (optional)

Once you have obtained the lepidopter signing key, you should verify the fingerprint of this key very carefully by obtaining copies of the fingerprint from trustworthy independent sources and comparing them to the downloaded key’s fingerprint to ensure they match. You could hen set the key’s trust level to “ultimate” (if you are satisfied with the verification process), so that you are not getting any warnings about untrusted signatures:

gpg --edit-key 0xBA56AC5A53E9C7A4

pub  4096R/0xBA56AC5A53E9C7A4  created: 2016-05-22  expires: never       usage: SC
                               trust: unknown      validity: unknown

gpg> fpr
pub   4096R/0xBA56AC5A53E9C7A4 2016-05-22 Lepidopter Team (signing key)
 Primary key fingerprint: 6255 1196 8E24 0F24 F6CF  D0B6 BA56 AC5A 53E9 C7A4

gpg> trust
pub  4096R/0xBA56AC5A53E9C7A4  created: 2016-05-22  expires: never       usage: SC
                               trust: unknown      validity: unknown
sub  4096R/0x566AA65E9738B6C8  created: 2016-05-22  expires: never       usage: E
[unknown] (1). Lepidopter Team (signing key)

Please decide how far you trust this user to correctly verify other users' keys
(by looking at passports, checking fingerprints from different sources, etc.)

  1 = I don't know or won't say
  2 = I do NOT trust
  3 = I trust marginally
  4 = I trust fully
  5 = I trust ultimately
  m = back to the main menu

Your decision? 5
Do you really want to set this key to ultimate trust? (y/N) y

pub  4096R/0xBA56AC5A53E9C7A4  created: 2016-05-22  expires: never       usage: SC
                               trust: ultimate      validity: ultimate
sub  4096R/0x566AA65E9738B6C8  created: 2016-05-22  expires: never       usage: E
[ultimate] (1). Lepidopter Team (signing key)

gpg> quit

Get the Lepidopter image signature

Download the lepidopter image signature.

Verify the Lepidopter image

In order to verify the lepidopter image both the image archive (.zip or .asc) and the signature file (.asc) should be within the same directory.

Verify that the compressed image lepidopter-v1.0.0-armel.img.zip matches its signature lepidopter-v1.0.0-armel.img.zip.asc by running:

gpg -v --verify lepidopter-v1.0.0-armel.img.zip.asc

You should look for the message Good signature from "Lepidopter Team (signing key)":

gpg: assuming signed data in `lepidopter-v1.0.0-armel.img.zip'
gpg: Signature made Fri 05 May 2017
gpg:                using RSA key 0xBA56AC5A53E9C7A4
gpg: using PGP trust model
gpg: Good signature from "Lepidopter Team (signing key)"
gpg: binary signature, digest algorithm SHA512

If you want to learn more about digital signatures and key verification read the documentation from Qubes OS.

Setting up your Raspberry Pi with Lepidopter

Depending on whether you are a Windows, Linux or Mac OS X user, please refer to the relevant section of the guide below on how to set up your Raspberry Pi with Lepidopter.

Linux users

If you are using Ubuntu and hesitate to use the terminal, you can use Disk Image Writer (graphical interface) to copy the Lepidopter image to an SD card. Alternatively, you can do this through your terminal.

Below we provide steps on how to copy the Lepidopter image to an SD card via Disk Image Writer or a terminal.

Disk Image Writer

Step 1. Download the Lepidopter image.

Step 2. Right-click on the downloaded Lepidopter image and select [Extract here] to extract the image.

Lepidopter-ubuntu

Lepidopter-ubuntu

Step 3. Install Disk Image Writer from the Ubuntu Software Center.

Step 4. Insert your SD card into your SD card reader, and subsequently connect your SD card reader to your laptop/desktop.

Step 5. Right-click on the extracted Lepidopter image and open it with Disk Image Writer.

Lepidopter-ubuntu

Step 6. Select your SD card reader within the drop-down menu.

Lepidopter-ubuntu

Important: Ensure you’re selecting your SD card reader, as (accidentally) selecting your hard drive could result in its corruption.

Step 7. Click [Start Restoring…]. This will copy the Lepidopter image to your SD card.

Lepidopter-ubuntu

Step 8. Click [Restore].

Lepidopter-ubuntu

Step 9. Type in your system password to authenticate this.

Lepidopter-ubuntu

Wait for the Lepidopter image to get copied to your SD card…

Lepidopter-ubuntu

Step 10. Eject when it’s done.

Lepidopter-ubuntu

Now your SD card contains OONI’s software!

Step 11. Insert your SD card into the [MICRO SD CARD] slot of your Raspberry Pi.

Step 12. Connect your Raspberry Pi to power and to the internet (with an Ethernet cable).

That’s all! From now on, your Raspberry Pi will automatically run OONI Probe every day at 6am UTC. The measurements will be sent directly to OONI’s server, where they will be processed and published on OONI Explorer (generally) within 24 hours.

Terminal

Please note that the use of the “dd” tool can overwrite any partition of your machine. If you specify the wrong device in the instructions below you could delete your primary Linux partition. Please be careful.

Step 1. Download the Lepidopter image.

Step 2. Extract the image:

unzip lepidopter-v1.0.0-armel.img.zip

If you have downloaded the xz compressed image:

xz --decompress --verbose --no-sparse lepidopter-v1.0.0-armel.img.xz

Step 3. Run df -h to see which devices are currently mounted.

Step 4. If your computer has a slot for SD cards, insert the card. If not, insert the card into an SD card reader, then connect the reader to your computer.

Step 5. Run df -h again.

The device that wasn’t there last time is your SD card. The left column gives the device name of your SD card. It will be listed as something like “/dev/mmcblk0p1” or “/dev/sdd1”. The last part (“p1” or “1” respectively) is the partition number, but you want to write to the whole SD card, not just one partition, so you need to remove that part from the name (getting for example “/dev/mmcblk0” or “/dev/sdd”) as the device for the whole SD card.

Note: The SD card can show up more than once in the output of df: in fact it will if you have previously written a Raspberry Pi image to this SD card, because the Raspberry Pi SD images have more than one partition.

Step 6. Now that you’ve noted what the device name is, you need to unmount it so that files can’t be read or written to the SD card while you are copying over the SD image. So run the command below, replacing “/dev/sdd1” with whatever your SD card’s device name is (including the partition number):

umount /dev/sdd1

If your SD card shows up more than once in the output of df due to having multiple partitions on the SD card, you should unmount all of these partitions.

Step 7. In the terminal copy the image to the card with this command, making sure you replace the input file if= argument with the path to your .img file, and the “/dev/sdd” in the output file of= argument with the right device name (this is very important: you will lose all data on the hard drive on your computer if you get the wrong device name). Make sure the device name is the name of the whole SD card as described above, not just a partition of it (for example, sdd, not sdds1 or sddp1, or mmcblk0 not mmcblk0p1).

dd bs=4M if=~/lepidopter-v1.0.0-armel.img of=/dev/sdd

Please note that block size set to 4M will work most of the time. If not, please try 1M, although 1M will take considerably longer.

Note that if you are not logged in as root you will need to prefix this with sudo.

The dd command does not give any information of its progress and so may appear to have frozen. It could take more than five minutes to finish copying to the card. If your card reader has an LED it may blink during the write process. To see the progress of the copy operation you can run pkill -USR1 -n -x dd in another terminal (prefixed with sudo if you are not logged in as root). The progress will be displayed (perhaps not immediately, due to buffering) in the original window, not the window with the pkill command.

Step 8. As root run the command sync or if a normal user run sudo sync (this will ensure the write cache is flushed and that it is safe to unmount your SD card).

Step 9. Remove the SD card from your card reader, insert it in the Raspberry Pi, and have fun!

Mac OS X users

You can copy the Lepidopter image to your SD card through the following steps:

Step 1. Download the Lepidopter image.

Step 2. Double-click on the downloaded Lepidopter image to extract it.

Step 3. Download ApplePi-Baker.

ApplePi-Baker window

ApplePi-Baker is a free utility that allows you to copy an IMG file to an SD card with only a few simple clicks.

Step 4. Insert your SD card into your SD card reader, and subsequently connect your SD card reader to your laptop/desktop.

Step 5. Start ApplePi-Baker.

Step 6. Select the Lepidopter IMG file in the Restore Backup section.

Step 7. Select your SD card reader (in the Pi-Crust section) as a location to copy the Lepidopter image to.

Important: Ensure you’re selecting your SD card reader, as (accidentally) selecting your hard drive could result in its corruption.

Step 8. Click [Restore Backup].

Wait for the Lepidopter image to get copied to your SD card…

Step 9. Type in your system password to authenticate this.

Step 10. Eject when it’s done.

Step 11. Insert your SD card into the [MICRO SD CARD] slot of your Raspberry Pi.

Step 12. Connect your Raspberry Pi to power and to the internet (with an Ethernet cable).

That’s all! From now on, your Raspberry Pi will automatically run OONI Probe every day at 6am UTC. The measurements will be sent directly to OONI’s server, where they will be processed and published on OONI Explorer (generally) within 24 hours.

Optimization

In OS X each disk may have two path references in /dev:

On a Class 4 SD card the difference was around 20 times faster using the rdisk path.

Windows users

You can copy the Lepidopter image to your SD card through the following steps:

Step 1. Download the Lepidopter image.

Step 2. Extract the image file from the downloaded .xz file, so you now have “lepidopter-v1.0.0-armel.img”.

Step 3. Download Win32DiskImager and extract the file.

Win32DiskImager window

Step 4. Insert the SD card into your SD card reader and check what drive letter it was assigned. You can easily see the drive letter (for example G:) by looking in the left column of Windows Explorer.

Step 5. Run Win32DiskImager.

You might need to run Win32DiskImager as Administrator. In this case, right- click on the file, and select ‘Run as Administrator’.

Step 6. Select the Lepidopter image file that you extracted above.

Step 7. Select the drive letter of the SD card in the device box.

Important: Ensure you’re selecting your SD card reader, as (accidentally) selecting your hard drive could result in its corruption.

Step 8. Click [Write].

Wait for the Lepidopter image to get copied to your SD card…

Step 9. Exit the imager and eject the SD card.

Step 10. Insert your SD card into the [MICRO SD CARD] slot of your Raspberry Pi.

Step 11. Connect your Raspberry Pi to power and to the internet (with an Ethernet cable).

That’s all! From now on, your Raspberry Pi will automatically run OONI Probe every day at 6am UTC. The measurements will be sent directly to OONI’s server, where they will be processed and published on OONI Explorer (generally) within 24 hours.

Note: In Windows, the SD card will appear to only have a fairly small size once written - about 55 to 75 MB. This is because most of the card has a partition that is formatted for the Linux operating system that the Raspberry Pi uses which is not visible in Windows. If you don’t see a small directory with files such as kernel.img then the copy may not have worked correctly.

Using flashnul (if Win32DiskImager isn’t successful)

You might not be able to choose the device in Win32DiskImager on some notebooks, so this is a different way to achieve the same thing on a Windows machine.

Step 1. Download the Lepidopter image.

Step 2. Extract the image file from the downloaded .xz file, so you now have “lepidopter-v1.0.0-armel.img”.

Step 3. Download flashnul.

The English translated version can be found here with a link to instructions (also translated).

Step 4. Insert the SD card into your SD card reader and check what drive letter it was assigned. You can easily see the drive letter (for example G:) by looking in the left column of Windows Explorer.

Step 5. Extract the application from the archive.

Step 6. Click Start button > All Programs > Accessories > Command Prompt, right click on it and Run as Administrator.

Step 7. Run flashnul with argument “-p”: