How to use Basler Industrial Camera (Pylon SDK) with OpenCV in Qt?

Introduction

Basler industrial cameras are widely used in the industry, Pylon SDK is provided by Basler to configure the camera and grab frame, but it lacks the abilty to perfrom image processing, therefore we will use OpenCV instead, Qt can provide a solid and robust UI cability.

Requirements

Hardware:

Basler ACE USB 3.0 Camera
PC with USB3.0 port

Software:

Ubuntu 18.04
Qt 5.12.0
OpenCV-4.0.0 or newer 4 (Please refer to this post on how to install/use OpenCV 4 with Qt.)
Pylon 5.1.0 SDK

Preparation

Installation

Download SDK file pylon_5.1.0.12682-deb0_amd64.deb from the official Pylon website. You should download the suitable version to match with your PC.

Make it excutable:

chmod +x pylon_5.1.0.12682-deb0_amd64.deb

Then double click the deb file, follow the guide to install it by default.

Configuration

To use Pylon C++ API, add the following code in .pro file of the Qt project:

# /opt/pylon5 is SDK installation path, please check your installation
INCLUDEPATH += /opt/pylon5/include  
QMAKE_LFLAGS += -Wl, --enable-new-dtags -Wl, -rpath, /opt/pylon5/lib64
LIBS += -L/opt/pylon5/lib64
LIBS += -Wl, -E
LIBS += -lGCBase_gcc_v3_1_Basler_pylon_v5_1
LIBS += -lGenApi_gcc_v3_1_Basler_pylon_v5_1
LIBS += -lgxapi
LIBS += -lpylonbase
LIBS += -lpylonutility
LIBS += -luxapi
LIBS += -lpylon_TL_usb
LIBS += -lMathParser_gcc_v3_1_Basler_pylon_v5_1
LIBS += -lNodeMapData_gcc_v3_1_Basler_pylon_v5_1
LIBS += -llog4cpp_gcc_v3_1_Basler_pylon_v5_1
LIBS += -lLog_gcc_v3_1_Basler_pylon_v5_1
LIBS += -lXmlParser_gcc_v3_1_Basler_pylon_v5_1

Include the following header in the corresponding header file:

#include <pylon/PylonIncludes.h>
// Namespace for using pylon objects.
using namespace Pylon;
// Settings for using Basler USB cameras.
#include <pylon/usb/BaslerUsbInstantCamera.h>

typedef Pylon::CBaslerUsbInstantCamera Camera_t;
// grabbed frame data from Pylon
typedef Camera_t::GrabResultPtr_t GrabResultPtr_t;

using namespace Basler_UsbCameraParams;

Code Usage

To make it easy to use, I wrote a FrameGrabber QObject Class. By using this class, program can easily call a function to configure, start, stop the camera, grab a frame data and return as a normal OpenCV Mat image data. Click to see framegrabber.h and framegrabber.cpp on how to use it.

Now I will pick some key points to introduce.

Initialize the camera:

void initCamera(){
    Pylon::PylonInitialize();
    Camera_t *pylonCamera;
    pylonCamera = new Camera_t;
}

Configure the camera, camera parameter file is here.

void configureCamera(){
    if (pylonCamera->IsGrabbing()) pylonCamera->StopGrabbing();

    try{
        // Method 1:
        // Load pfs file into the camera's node map with enabled validation.
        const char cameraParaFile[] = "../conf/acA2440-35uc_22776933.pfs";
        //CFeaturePersistence::Save(cameraParaFile, &pylonCamera->GetNodeMap());
        CFeaturePersistence::Load(cameraParaFile, &pylonCamera->GetNodeMap(), true);

        // Method 2:
        // The parameter MaxNumBuffer can be used to control the count of buffers
        // allocated for grabbing. The default value of this parameter is 10.

        //pylonCamera->MaxNumBuffer = 10;
        /*
        pylonCamera->ExposureAuto.SetValue(ExposureAuto_Off);
        pylonCamera->ExposureMode.SetValue(ExposureMode_Timed);
        pylonCamera->ExposureTime.SetValue(25000); // us

        pylonCamera->AcquisitionFrameRateEnable.SetValue(true);
        pylonCamera->AcquisitionFrameRate.SetValue(35.0);

        pylonCamera->Width.SetValue(2048);
        pylonCamera->Height.SetValue(1536);
        
        pylonCamera->CenterX.SetValue(true);
        pylonCamera->CenterY.SetValue(true);
        */
        // Warm up camera for 0.1s
        usleep(100000);
    }
    catch (GenICam::GenericException &e){
        // Error handling.
        cerr << "[ERROR!] An Exception Occurred in configureCamera()."
            << endl << e.GetDescription() << endl;
    }
}

Scan available Pylon camera devices:

void scanDevices(){
    // Get all the connected camera
    Pylon::DeviceInfoList_t cameraList;
    Pylon::CTlFactory::GetInstance().EnumerateDevices(cameraList, true);
    QString cameraNameString = NULL;
    if (cameraList.size() != 0)
    {
        // first camera
        cameraNameString = QString(cameraList[0].GetFriendlyName());
    }
}

Connect to the camera:

void connectCamera(){
    try{
        // Attach connected camera to the pylon camera object
        pylonCamera->Attach(CTlFactory::GetInstance().CreateFirstDevice());
        // Open the camera, but not yet start grabbing
        pylonCamera->Open();
        // Warm up camera for 0.1s or 100ms
        usleep(100000);
        if(pylonCamera->IsOpen()){
            qDebug() << "Successful to connect camera.";
            configureCamera();
        }
        else{
            qDebug() << "FAIL to connect camera.";
        }
    }
    catch (GenICam::GenericException &e){
        // Error handling.
        cerr << "[ERROR!] An Exception Occurred in connectCamera()."
             << endl << e.GetDescription() << endl;
    }
}

Start grabbing the frame:

void startGrabbing(){
    try{
        pylonCamera->StartGrabbing(GrabStrategy_LatestImageOnly);
    }
    catch (GenICam::GenericException &e){
        // Error handling.
        cerr << "[ERROR!] An Exception Occurred in startCapture()."
                << endl << e.GetDescription() << endl;
    }
    // send a signal to get plyon frame, and convert it to OpenCV Mat, readFrame() is in next part.
    emit signalToSent(readFrame());
}

Convert the pylon frame data to OpenCV Mat data:

cv::Mat readFrame(){
    try{
        CPylonImage pylonFrame;
        CImageFormatConverter formatConverter;
        formatConverter.OutputPixelFormat = PixelType_BGR8packed;
        // This smart pointer will receive the grab result data.
        GrabResultPtr_t ptrGrabResult;
        // Wait for an image and then retrieve it. A timeout of 5000 is used.
        pylonCamera->RetrieveResult(5000, ptrGrabResult, TimeoutHandling_ThrowException);

        if (ptrGrabResult->GrabSucceeded()){
            // Convert the grabbed buffer to pylon image
            formatConverter.Convert(pylonFrame, ptrGrabResult);
            // Create an OpenCV image out of pylon image
            cv::Mat tempFrame = cv::Mat(ptrGrabResult->GetHeight(), ptrGrabResult->GetWidth(),
                                        CV_8UC3, (uint8_t *) pylonFrame.GetBuffer());
            // cvFrame is globally declared
            cvFrame = tempFrame.clone();

            return cvFrame;
        }
        else{
            qDebug() << "Fail to Grab Frame.";
        }
    }
    catch (GenICam::GenericException &e){
        // Error handling.
        cerr << "[ERROR!] An Exception Occurred in readFrame()."
             << endl << e.GetDescription() << endl;
    }
}

Stop Grabbing the frame:

void stopGrabbing(){
    if (pylonCamera->IsGrabbing()) pylonCamera->StopGrabbing();
}

Disconnect the camera:

void disconnectCamera()
{
    if (pylonCamera->IsOpen()){
        if (pylonCamera->IsGrabbing())  pylonCamera->StopGrabbing();
        pylonCamera->Close();
    }
    pylonCamera->DetachDevice();
}

Introduction#

Requirements#

Preparation#

Installation#

Configuration#

Code Usage#