Introduction

VideoLat is a tool to help you analyse video delays, mainly aimed at conferencing applications. Basically, it works by generating a barcode on-screen and then measuring how long it takes until that same barcode is detected by the camera. This method of measurement also takes into account delays caused by camera, grabber hardware, video output card and the video display. VideoLat runs on separate hardware from the conferencing application you want to test, usually a MacBook, an iPhone or an iPad.

Because you need to calibrate your measurement machine, the normal videoLat workflow is:

  1. Obtain a calibration of your videoLat device,
  2. Run a measurement of your system-under-test,
  3. View and analyse the results, and export them.

VideoLat measurements can be separated into two broad categories that have different requirements in hardware (one or two videoLat devices) and calibration.

Roundtrip measurements

Roundtrip measurements are done using a single copy of videoLat. videoLat displays a QR-code (or plays a sound) which should then be picked up by the conferencing camera, and transmitted to the remote conferencing application. Here, it should be picked up from the conferencing screen by the remote conferencing camera (either by camera positioning or by using a mirror) and transmitted back to the local conferencing application. Here the videoLat camera should pick up the image again.

Roundtrip measurements require a roundtrip-calibration of your videoLat device. Such a calibration run is done by holding up a mirror in front of the videoLat camera and screen. Alternatively, if you have a movable camera such as a separate USB camera you can simply point the camera at the screen. A roundtrip calibration measures the total delay of the videoLat camera and the videoLat screen, and this delay is remembered and subtracted from future roundtrip messurements.

One-way measurements

One-way measurements are done using two copies of videoLat, one running as Sender-Server and another running as Receiver-Client. The Sender-Server displays a QR-code which should be picked up by the conferencing system camera and transmitted to remote. There, the displayed QR-code is detected by the Receiver-Client videoLat, which communicates it back to the Sender-Server videoLat over the network.

A one-way measurement requires two calibrations: a screen-calibration of the sender videoLat device and a camera calibration of the receiver videoLat device.

Roundtrip or one-way?

From a user experience point of view the two measurements are very similar (but do remember to multiply or divide the result by 2 when comparing them). The difference is in the convenience of taking the measurement.

One-way measurements are much easier to take: you hold up one device screen in front of the videoconferencing system and on the remote end you point your camera at the screen. Roundtrip measurements require careful positioning of cameras and screens (and a steady hand!) which can be especially difficult with cameras that are fixed to the screen, such as on an iPhone or when using the iSight on a MacBook.

On the other hand a roundtrip calibration is easy, but the two calibrations for one-way measurements are more difficult to take. Think about it: you cannot calibrate only a camera or only a screen, you always need both a camera and a screen. The workaround is that you calibrate using a device with a known delay. Usually this will be a second iPhone or iPad for which the delays are already known: you now calibrate your new device against this known device, both old-screen-to-new-camera and new-screen-to-old-camera, and now you have your calibrations.

For the Mac version of videoLat there is another option for doing a camera-only or screen-only calibration, but you need to build dedicated hardware using an Arduino.

Calibration

The easiest way to obtain a calibration is to download it from videoLat.org, provided someone with exactly the same hardware as you has uploaded their calibrations previously.

On the Mac, select File, New Measurement... and the Download Calibration tab. The popup menu will show all calibrations available online for your cameras and screens (excluding ones you have downloaded earlier). Select one, press Download Selected. Repeat for all your devices.

On iPhone or iPad, select Download Calibration in the home screen. Selecting an individual calibration will download and display it. Return to the home screen and repeat for both cameras and your screen.

These calibrations are community-supplied, so you should inspect them and possibly do a test measurement (for example using a mirror) before relying on them. If calibrations for your iPhone/iPad, or for the combination of your mac, camera and screen are not available online you must run your own calibrations, see the next sections. In this case, it would be nice if you shared your calibrations online, for the benefit of other videoLat users in the future.

Round trip calibrations

If you need to do your own roundtrip calibration you need a mirror, so your camera can see your screen. Alternatively, on a Mac with a USB camera, you point the camera directly at the screen.

On the Mac, select File, New Measurement... and the New Measurement tab. Select Video Roundtrip Calibrate from the popup and press Measure.

A New Video Roundtrip Calibration window appears, position it on the screen you want to use for your calibration. Next, you select the camera you want to use as input. Output from the selected camera is shown in the preview area. Use this to position your camera (and window!) so that it has a clear view of the blue square, with not too much distortion. Using a mirror is possible, rotation is no problem, and a fair bit of skewing can be handled too.

Now you press Prepare to start the calibration preparation process.

On iPhone or iPad, select Run New Calibration in the home screen. Select Video Roundtrip Calibrate. Select the camera to use, which will probably be the front camera in case of a roundtrip calibration. Use the preview window to position the mirror so that the iPhone camera gets a good view of the iPhone screen. Make sure things are held reasonably steady: you will have no camera preview during the rest of the calibration.

Press continue to start the calibration preparation process.

Preparation will show a couple of QR-codes and determine mirroring, approximate delays and such. After the preparation phase you should refrain from changing the physical setup. If the preparation phase is successful the Run button will be enabled. If it is not successful you may have to re-select your measurement type or camera to try again.

Now you press Run and the real measurement will start. In the data area you will see QR-codes, and in the Measurement Run area you will see how many succesful measurements have been taken and current average delay and standard deviation. When you are happy (or bored) press Stop and the data view window (below) will open.

Determining the number of measurements to be taken is a bit of a black art. For the calibrations you want a distribution of measurements that is relatively close to a normal distribution, because it makes your real measurements more trustworty. Start with 500 or 1000 samples, and inspect the distribution in the data view window.

Here is a short video showing how to do a roundtrip calibration of the front camera of an iPhone:

And here is an instruction video for doing a Mac roundtrip calibration, but note that this video is for videoLat 1.0, so the details on how to start the calibration have changed slightly:

One-way calibrations

To do a one-way calibration of your camera you need a second device running videoLat for which you already have the screen calibration. You make sure both devices are on the same WIFI network. The calibrated device will act as Sender, and you point the camera of the device to calibrate, the receiver, at the sender screen. The sender will show an initial QR-code that allows the receiver to contact the sender over the WIFI network. Then, during the calibration measurement the receiver will communicate arrival times to the sender. Because the sender screen is already calibrated the two videoLat copies can now compute the delay of the receiver camera.

A one-way calibration of your screen is very similar, but with sender and receiver roles switched. So, to summarize, there are four measurement modes involved in doing a one-way calibration:

  • Camera Calibrate using Remote Calibrated Screen (Slave,Client): calibrate the camera of this device
  • Video Transmission (Master,Server): calibrated helper device for calibrating your camera
  • Screen Calibrate using Remote Calibrated Camera (Master,Server): calibrate the screen of this device
  • Video Reception (Slave, Client): calibrated helper device for calibrating your screen

The two procedures are very similar, the only difference being which device you are calibrating, Here is a list of the steps to follow:

  1. Connect both devices to the same WiFi network.
  2. Start the sender device in the right mode.
  3. Go through the steps until the initial QR code is shown.
  4. Start the receiver device in the right mode, do not point the camera yet.
  5. Go through the steps of selecting camera and such.
  6. Point the camera at the sender screen. You will see the mode on both devices go to Determining RTT and eventually the blue screen is shown again
  7. Press start on the sender device, and run a few hundred measurements
  8. Press stop on the sender device, and both devices will show the result. The new device will interpret it as a calibration measurement.

Here is a video showing the procedure with two iPhones:

Measurement View

The measurement view is shown after running a calibration or a measurement, or after opening or downloading a calibration or measurement document. If the view is shown because you did a new calibration you may be asked to share this calibration on videolat.org, so other videoLat users can download it in the future, in stead of having to do the calibration themselves.

The layout of the view is slightly different on Mac and iOS but contains the same information. On the top there are the results: average delay and standard deviation over all measurements. Also included is metadata such as time, date and location of the measurement, and a description of the input and output device used. Next there is a distribution plot of all measurements, with aggregate distribution of your data points and aggregate normal distribution also shown. At the bottom are the delays for the individual measurements.

On the Mac, there are buttons to open the calibration measurement(s) on which this measurement depends (if any), and the File menu has Save, Print and Export commands (see below).

On iOS, there is an action button that allows you to print, email, upload export or delete your measurement or calibration.

Round trip measurement

After you have obtained a roundtrip calibration you are ready to take a measurement of the system under test. Open a new document, but this time select Measurement type Video Roundtrip.

Again, you select the camera, but now you also select the base calibration run you created in the previous step.

Now comes the difficult bit, you need to make sure all three cameras look at the right screen, and after the QR code has travelled through the whole chain of cameras and displays it must still be recognizable:

  • The measurement machine camera looks at the system-under-test screen,
  • The system-under-test camera looks at the measurement machine screen,
  • The remote system-under-test camera looks at its own screen.

An order of positioning that tends to work reasonably well is as follows:

  1. Set the system-under-test to show a self view, and position the system-under-test camera and the measurement machine screen so that you get a good picture of the dark blue rectangle on the system-under-test self view.
  2. Set the system-under-test so it shows a view from the remote system-under-test. Point the remote system-under-test camera at its own screen (or at a mirror so it can see its own screen). Position things so you again see a decent image of the blue rectangle, but now after it has been sent back-and-forth to the remote system-under-test.
  3. Position the measurement system camera so it can see the blue rectangle on the system-under-test screen. Confirm in the videoLat preview window that you get a decent picture.

After this positioning, follow the same procedure as for calibration: Prepare, Run, Stop. The data view will open again.

And here is a short instruction video for videoLat 1.0 giving step-by-step instructions on doing a roundtrip video delay measurement (but note that the exact steps for starting the measurements have changed slightly in videoLat 2.0):

Exporting results

When you have the data view open on the Mac you can use the Export as CSV... command to save all the data from the measurement run as CSV (text file with lines of comma-separated values). You get three output files which you can import into a spreadsheet or plotting program or other external tool. on iPhone or iPad you can use the action button and email the CSV documents.

The description file has two columns, "key" and "value". This file contains the metadata of the measurement run, such as measurement type and devices used.

The measurements file contains the raw measurements. It has three columns:

  • "at" is the time of the measurement, in microseconds, since some unspecified epoch.
  • "data" is a string, representing the image (or sound) generated at that time
  • "delay" is the delay in microseconds until the image/sound was detected again

The distribution file contains a frequency distribution of the measurements, split over 100 bins. It is for convenience only, it can be generated from the measurements file, but in Numbers or Excel this can take an inordinate amount of time if you have many data points. It has three columns:

  • "lowerBound" is the lower bound of the bin
  • "upperBound" is the upper bound of the bin (which is the lower bound of the next one)
  • "binValue" is the fraction of the measurements that fall into this bin.