Controlling a Spatial Light Modulator remotely using a Raspberry Pi

Standard phase spatial light modulators (SLMs) have the advantage of being controlled as a secondary display. It saves money on dedicated control interfaces and simplifies the usage. However, when trying to control an experiment remotely, which is especially needed these days, It adds some complexity when using remote desktop software or ssh. I will detail here how to use a Raspberry Pi to control an SLM and send images from a computer on the same local network. The computer that controls the experiment can now easily be controlled from the comfort of your home.

The problem

Let's say you have a lab computer that controls your experiment and you want to control it from home. When trying to control the lab computer using a remote desktop program, the information is displayed on your home computer, not on the monitor(s) in the lab. But SLMs are controlled as secondary displays, so they stop displaying anything. While I have few ideas on how to overcome this on Linux using ssh, I did not find any good solution on Windows. Moreover, I wanted for quite some time to have a way to control an SLM independently from the operating system, i.e. as a typical hardware device: you use a library that controls the device and that's it. One disadvantage of using an SLM as a display is that it can be perturbed by the operating system, for instance, when you hit the Window key on your keyboard, it grays out all the displays, including the SLM. You can also mess up things if you drag a window inadvertently on the area that controls the SLM, or slide your mouse cursor on that area. Finally, you are limited by the number of video connectors on your computer if you want to control several SLMs.

The setup

The typical setup is represented in Fig. 1. Your lab computer and your Raspberry Pi are connected to the same local network (or subnetwork, typically with a switch, to prevent people in the lab to interact with your devices). You then connect from your home to your lab computer through a gateway using ssh or a remote desktop protocol.

Figure 1. Schematic of the setup

To make it work, we need the Raspberry Pi, which is connected to the SLM, to work as a server. While it should work on any model, I strongly recommend using a Raspberry Pi 4, as it is the only one with a full-speed Gigabit Ethernet port. Otherwise, you may end up with slow communication that will impact the maximal refresh rate you can use on the SLM. It will listen to a given port and display images on the SLM when it receives them. The lab computer will act as a client, sending information to the Pi using TCP protocol. Conveniently, I implemented these features in the slmPy Python module (see tutorial here and Github page there).

Configuring the Raspberry Pi

If you do not care how it works, you can do everything in one shot using the script I wrote. To download it and execute it, you need to be connected to the internet and run the following command (directly on the Raspberry Pi or connected via ssh):

wget https://raw.githubusercontent.com/wavefrontshaping/slmPy/master/rpi_server/install.sh
bash install.sh

If you do that, just skip the following and go to the Use the SLM section.

First, we need to install slmPy, for that you need pip and the required dependencies:

# install pip
sudo apt-get install python3-pip -y

# install wxPython (required for slmPy)
sudo apt install python3-wxgtk4.0 -y
sudo python3.7 -m pip install wxPython

# install slmPy
git clone git@github.com:wavefrontshaping/slmPy.git
cd slmPy
python3.7 setup.py install

We also want to deactivate the screen saver and the sleep behavior:

sudo bash -c 'echo -e "\n[SeatDefaults]\nxserver-command=X -s 0 -dpms" >> /etc/lightdm/lightdm.conf'

Now, copy the script that will run the server code on the Raspberry Pi in your home folder:

cp rpi_server/server.py ~

If you want the Raspberry Pi to run this script automatically upon startup:

mkdir -p /home/pi/.config/autostart
cp rpi_server/SLM.desktop /home/pi/.config/autostart/

If you want to run the server manually, connect through ssh, but before running the code, you need to gain access to the display

export DISPLAY=:0
python3.7 server.py

Use the SLM

From your lab computer, controlled remotely or locally, you can send images on the SLM using slmPy:

import slmpy

from slmpy import Client
import numpy as np
import time

# Port number, should be the same port as in server.py on the Pi
PORT = 9999 
# ip adress or name of the Raspberry Pi
SERVER = '10.0.0.0'

# resolution, should be the same as the SLM
resX, resY =  (1280, 720)

# connect to the Raspberry Pi
client = slmpy.Client()
client.start(SERVER, port = PORT)

# create an image to display
X,Y = np.meshgrid(np.linspace(0,resX,resX),np.linspace(0,resY,resY))
frame = np.round((2**8-1)*(0.5+0.5*np.sin(2*np.pi*X/50))).astype('uint8')

t0 = time.time()
client.sendArray(frame)
print(f'Sent image in {time.time()-t0:.2f} seconds')

# close communication with the Raspberry Pi
client.stopServer()

And you are done!

To go further

Sending images, especially for HD devices, can take some time. The solution I use under the hood is to compress the images before sending them and then decompressing them on the Raspberry Pi. A trade-off should be found as a too high compression rate will give rise to a long decompression time on the Raspberry Pi. By default, slmPy uses the zlib module, but you can also choose gzip or bz2 and specify a compression level. The higher the value, the higher the compression rate, read the doc of the corresponding modules to learn more.

On the server's side:

slm.listen_port(port = PORT, compression = 'bz2')

On the client's side:

Client.start(..., compression = 'bz2', compression_level = 2)         

By default, the client waits for the server to display the image and to send back a confirmation message before returning. This ensures that all the images are displayed. For some purposes, you may not care about dropping some frames, for instance, if you want real-time applications. To change this behavior:

Client.start(..., wait_for_reply = False)