Connecting an MLX90640 Thermosensor with Numpy/OpenCV

I recently got my hands on an MLX90640 thermal sensor. Sparkfun has a good tutorial to get started. However, I wanted to use Numpy and OpenCV to toy around with the data instead of the Processing IDE. I took their example and wrote a Python script that replaces the Processing IDE and loads the data into Numpy and OpenCV.

A few seconds in I hold my soldering iron over the sensor, and you can see the colormap adjust to the temperature range dynamically.

Prerequisites

Hardware

• Teensy by PJRC -- I have a Teensy 4.0, but other versions should also work
• MLX90640: I have a breakout board from Seeed Studio. Other breakout boards also work, or using the MLX90640 directly on a breadboard should be fine.
• Serial to USB converter

Software

All the software I use is open source and free. Besides Python 3, I use the following packages:

• numpy
• opencv-python
• pyserial

To program the Teensy you can use the Arduino IDE or -- what I prefer -- PlatformIO in VSCode.

Hardware Setup

The MLX90640 has an I2C interface. Connect the SCL and SDA pins of the Teensy to the corresponding pins on the MLX90640 and add a pull-up resistor to 3.3 V (e.g. 10 kOhm). Also connect GND and 3.3 V power. Then connect the serial to USB converter to your PC. That's it for hardware.

Software Setup

The script can be found on GitHub. The folder Example2_OutputToProcessing contains the Sparkfun firmware for connecting to the Processing IDE. Open Example2_OutputToProcessing.ino in the Arduino IDE, compile, and flash it to the Teensy. Arduino's serial monitor should now show the raw data coming from the MLX90640.

To consume the data with Numpy, use the test.py script. Make sure to update the serial port on line 5.

import numpy as np
import serial, cv2, math

serialport = "COM34" # or something like '/dev/ttyS0'
scaling = 20

width = scaling * 32
height = scaling * 24

img = np.zeros([height,width,3])
imgGray = np.zeros([height,width,3])

try:
    ser = serial.Serial(serialport,115000)
except serial.SerialException:
    print("Cannot open serial port")
    quit()

try:
    print("press Ctrl-C to end")
    while True:
        # read data from serial port
        cc = str(ser.readline())
        cc = cc[2:-6]
        data = np.fromstring(cc, sep=',')

        # reshape data into matrix
        output = data.reshape(24,32)

        # scale to 0-255
        minValue = math.floor(np.amin(output))
        maxValue = math.ceil(np.amax(output))
        output = output - minValue
        output = output * 255 / (maxValue - minValue)

        # resize image
        dim = (width, height)
        output = cv2.resize(output, dim, interpolation=cv2.INTER_LINEAR)

        # apply colormap
        imgGray = output.astype(np.uint8)
        img = cv2.applyColorMap(imgGray, cv2.COLORMAP_JET)

        # put min/max text on image
        text = "Min: " + str(minValue) + " C  Max: " + str(maxValue) + " C"
        font = cv2.FONT_HERSHEY_SIMPLEX
        org = (20, 50)
        image = cv2.putText(img, text, org, font, 1, (255, 255, 255), 2, cv2.LINE_AA)

        cv2.waitKey(50)
        cv2.imshow("image", img)

except KeyboardInterrupt:
    print("Bye bye :)")
    ser.close()