In this post, I would like to share how to build a barcode scanner (for Android and iOS) in QML using OpenCV.
As a first step, we need to build the OpenCV library or download it from here: https://github.com/niqt/ocvdecoder (the readme discusses QR code scanning, but the library can also be used for barcode scanning).
To reach the goal we have to do:
- Create the C++ class
- Create the QML Item
- Use the QML component
C++
The logic involves receiving frames from the camera and decoding them until the barcode is recognized. These frames are captured by videoSync, which is the object responsible for obtaining frames from the camera.
/*
* This class implement a qml wrapper for the OpenCV bar code feature
* */
class Bardecoder : public QObject
{
Q_OBJECT
QML_ELEMENT
Q_PROPERTY(QObject* videoSink WRITE setVideoSink) // the video sync used to get the frames
Q_PROPERTY(bool run READ run WRITE setRun) // start/top the decode
public:
Bardecoder(QObject *parent = nullptr);
void setVideoSink(QObject *videoSink);
bool isDecoding() {return m_decoding; } // the status of the decoding - decondig in progress
void setRun(bool run);
bool run() {return m_run;}
public slots:
void setFrame(const QVideoFrame &frame);
signals:
void videoSyncChanged();
void decoded(const QString &code);
private:
QVideoSink *m_videoSink;
cv::Ptr<cv::barcode::BarcodeDetector> m_bardet;
QFuture<void> m_processThread;
bool m_decoding;
bool m_run;
};
The Bardecoder class is a QML_ELEMENT. It has two properties: videoSync (to set the videoSync) and run to initiate the decoding process.
The setFrame slot is invoked when videoSync captures a frame from the camera.
In the private section of the class, the key components are m_barded (the OpenCV barcode decoder) and m_processThread (the thread that executes the decoding action).
In the cpp part we have:
Bardecoder::Bardecoder(QObject *parent)
: QObject{parent}
{
m_bardet = makePtr<barcode::BarcodeDetector>("", ""); // create the bardetector
}
void Bardecoder::setVideoSink(QObject *videoSink)
{
m_videoSink = qobject_cast<QVideoSink*>(videoSink);
if (m_videoSink) {
QObject::connect(m_videoSink, &QVideoSink::videoFrameChanged, // when a frame is received call setFrame
this, &Bardecoder::setFrame);
}
}
void Bardecoder::setFrame(const QVideoFrame &frame)
{
if(m_run && !isDecoding() && m_processThread.isFinished()) { // if start decode and is not decoding and the previous is finished => decode the frame
m_decoding = true;
QImage image = frame.toImage().convertToFormat(QImage::Format_RGB32).rgbSwapped(); // convert the frame to an imageso can be used from opecv
m_processThread = QtConcurrent::run([=]() {
if(image.isNull()) {
m_decoding = false;
return;
}
cv::Mat img(image.height(), image.width(), CV_8UC4, (void *) image.bits(), image.bytesPerLine());
vector<Mat> points;
vector<Point> corners;
vector<cv::String> decode_info;
vector<string> decode_type;
m_bardet->detectMulti(img, corners);
m_bardet->detectAndDecodeWithType(img, decode_info, decode_type, corners);
if (decode_info.size() > 0) {
auto code = QString(decode_info.at(0).c_str());
if (code.length() > 0) // if the barcode is decoded emit the signal
emit decoded(QString(decode_info.at(0).c_str()));
}
m_decoding = false;
});
}
}
void Bardecoder::setRun(bool run)
{
m_run = run;
}
The two main components are setVideoSync, which sets the slot, and setFrame. When a frame changes, setVideoSync triggers setFrame.
Within setFrame, the decoding action occurs. This process begins if run is true (where run functions as a start command). The decoding action starts only if there is no other frame currently being decoded and if the previous decoding thread has finished. Before using the OpenCV decoder functions, it’s necessary to transform the frame into an image in a specific format. Once this transformation is complete, the OpenCV functions are called as detailed in the OpenCV documentation. If the barcode is successfully decoded, a signal is emitted.
QML
Create the BarReader QML component:
import QtQuick
import Felgo
import QtQuick.Controls 2.0
import QtQuick.Layouts 1.1
import QtMultimedia
import bardecoder
/*
This element contains the logic and the components for the Barcode reader.
It use the camera, the CaputureSession to capture the video from the camera and it sent
to VideoOutput to be shown. The decoder get the video frames using the videoSync from the VideoOutput.
The signal barCaptured is emitted if the decoder found a barcode.
*/
Item {
id: item
// This property enable/disable the decoding action
property alias run: decoder.run
// This signal is emitted when the barcode is found
signal barCaptured(string barcode)
Camera // The camera
{
id:camera
active: run
focusMode: Camera.FocusModeAuto
}
CaptureSession {
camera: camera
videoOutput: videoOutput
}
VideoOutput
{
id: videoOutput
anchors.fill: parent
// The follow properties are used to scale the red capture area on the camera video
property double captureRectStartFactorX: 0.25
property double captureRectStartFactorY: 0.25
property double captureRectFactorWidth: 0.5
property double captureRectFactorHeight: 0.5
MouseArea { // Clicking remove the autofocus
anchors.fill: parent
onClicked: {
camera.customFocusPoint = Qt.point(mouseX / width, mouseY / height);
camera.focusMode = Camera.FocusModeManual;
}
}
Rectangle { // The capture zone
id: captureZone
color: "red"
opacity: 0.2
width: parent.width * parent.captureRectFactorWidth
height: parent.height * parent.captureRectFactorHeight
x: parent.width * parent.captureRectStartFactorX
y: parent.height * parent.captureRectStartFactorY
}
}
Bardecoder { // The barcode decoder
id: decoder
videoSink: videoOutput.videoSink // in this way capture the video frames
onDecoded: code => {
console.log("QR " + code)
run = false // stop to decode
barCaptured(code) // send the value
}
}
}
The logic behind this process is detailed in the code comments. The only aspect not specified in these comments is that we use a pink (red with opacity)rectangle to define the area of the camera used for decoding.
Now, we can integrate it into our page in the following way:
BarReader {
id: isbnReader
visible: false
run: false
anchors.fill: parent
onBarCaptured: barcode => {
visible = false // hide itself
run = false // stop to scanning
}
}
A working example of this project can be found here: https://github.com/niqt/felgobar. Although it uses Felgo, the same principles apply to pure Qt as well.
