{"id":411,"date":"2023-05-09T22:23:35","date_gmt":"2023-05-09T13:23:35","guid":{"rendered":"https:\/\/www.drassal.net\/wp\/?p=411"},"modified":"2023-05-09T22:28:12","modified_gmt":"2023-05-09T13:28:12","slug":"creating-a-new-open-source-iot-platform-the-mqtt-server-broker","status":"publish","type":"post","link":"https:\/\/www.drassal.net\/wp\/creating-a-new-open-source-iot-platform-the-mqtt-server-broker\/","title":{"rendered":"Creating a new open source IoT platform,\u00a0the MQTT server\/broker"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

So far we have come up with the architecture and the database structure for the new IoT platform. Next we should address the MQTT communications portion. The MQTT protocol is proposed due to the publish\/subscribe features, lightweight overhead, and ability to encrypt\/secure the communications. It is possible to use HTTPS to post the data to the server but the overhead of headers and renegotiating the SSL encryption with new communication proves to be too much extra data to send. With HTTP posts the data also flows only one direction, from the sensor to the server. Reverse communication is not possible unless the connection is kept open.<\/p>\n\n\n\n

MQTT has the advantage of the communication being two way. The sensor (or gateway) can subscribe to one more more MQTT topics (or channels) and then receive any messages that are published to those topics. Addressing the topic (channel) names and such will be saved for a later topic. For this topic we would like to address now to setup the MQTT broker (server) that all the clients will connect to. For this we will propose the open source MQTT broker Mosquitto. The details on Mosquitto can be found at this link https:\/\/mosquitto.org<\/a>. There are installer packages available for various OSs but we have chosen to go about making a portable installation compiling from source, which is outlined below.<\/p>\n\n\n\n

The server platform used for this is Linux Rocky 9 (formally CentOS). First update the system, then following libraries will need to be installed in order to compile the source.<\/p>\n\n\n\n

Step 1: Update the system

sudo dnf update\nsudo dnf clean all<\/pre><\/p>\n\n\n\n
Step 2: Install GNU Compiler Collection (GCC) on Rocky Linux
sudo dnf group list\nsudo dnf clean all\nsudo dnf update\nsudo dnf groupinstall "Development Tools"<\/pre>If that command doesn’t work for you, run the command below.
sudo dnf group install "Development Tools"<\/pre><\/p>\n\n\n\n
Step 3: After successfully installing “Development tools,” run the command below to view the parts that come with this package.
dnf groupinfo "Development Tools"<\/pre><\/p>\n\n\n\n
Step 4: Check GCC Version and Installation Directory
Group: Development Tools\n Description: A basic development environment.\n Mandatory Packages:\n   autoconf\n   automake\n   binutils\n   bison\n   flex\n   gcc\n   gcc-c++\n   gdb\n   glibc-devel\n   libtool\n   make\n   pkgconf\n   pkgconf-m4\n   pkgconf-pkg-config\n   redhat-rpm-config\n   rpm-build\n   rpm-sign\n   strace\n Default Packages:\n   asciidoc\n   byacc\n   diffstat\n   git\n   intltool\n   jna\n   ltrace\n   patchutils\n   perl-Fedora-VSP\n   perl-generators\n   pesign\n   source-highlight\n   systemtap\n   valgrind\n   valgrind-devel\n Optional Packages:\n   cmake\n   expect\n   rpmdevtools\n   rpmlint<\/pre><\/p>\n\n\n\n
Step 5: Check if installation was successful.
gcc --version<\/pre>If everything installed correctly this should be returned indicating the GCC toolchain is now installed.
gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2)\nCopyright (C) 2021 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.<\/pre><\/p>\n\n\n\n
Now that the toolchain is installed we can create the directory structure for building.  Assuming we want to do this in our home directory, execute the following commands to install wget (used for downloading files later) and create the directory structure.
sudo dnf install wget\ncd\nmkdir mosquitto\ncd mosquitto<\/pre><\/p>\n\n\n\n
Next clone the cJSON source and build it, we will need to link to this when we compile Mosquitto.
git clone https:\/\/github.com\/DaveGamble\/cJSON.git\nmv cJSON cjson\ncd cjson\ngit checkout b45f48e\nmake<\/pre><\/p>\n\n\n\n
If the above link is dead or not working please download it from here and use these commands instead.
wget https:\/\/www.drassal.net\/filestore\/mosquitto_20230509\/cJSON_20230329_b45f8e.zip\nunzip cJSON_20230329_b45f8e.zip\nmv cJSON-master cjson\ncd cjson\nmake<\/pre><\/p>\n\n\n\n
Assuming this builds without error let’s now get the Mosquitto source and build it.  However, building from the GitHub source directly didn’t go smoothly with missing dependencies so we will use the source below.  First install the required dependencies.
sudo dnf install c-ares cjson libwebsockets openssl uthash<\/pre><\/p>\n\n\n\n
Now download and compile Mosquitto.
cd\ncd mosquitto\nwget https:\/\/www.drassal.net\/filestore\/mosquitto_20230509\/config.mk\nwget https:\/\/mosquitto.org\/files\/source\/mosquitto-2.0.15.tar.gz\ntar -xvzf mosquitto-2.0.15.tar.gz\ncp config.mk mosquitto-2.0.15\/.\ncd mosquitto-2.0.15<\/pre><\/p>\n\n\n\n
If the above link is broken or does not work the file can be downloaded from here instead.
wget https:\/\/www.drassal.net\/filestore\/mosquitto_20230509\/mosquitto-2.0.15.tar.gz<\/pre><\/p>\n\n\n\n
The following changes were made to config.mk, but are included in the above download.
# Build with SRV lookup support.\nWITH_SRV:=yes\n# Build with websockets support on the broker.\nWITH_WEBSOCKETS:=yes\n# Comment out to disable SSL\/TLS support in the broker and client.\n# Disabling this will also mean that passwords must be stored in plain text. It\n# is strongly recommended that you only disable WITH_TLS if you are not using\n# password authentication at all.\nWITH_TLS:=yes\n# Build with bundled uthash.h\nWITH_BUNDLED_DEPS:=yes\n# Build man page documentation by default.\nWITH_DOCS:=no<\/pre><\/p>\n\n\n\n
Finally kick off the build with the following command.  Replace “rocky” with your username so we point to the correct location for cJSON linking.
LDFLAGS+="-L\/home\/rocky\/mosquitto\/cjson\/" CFLAGS+="-I\/home\/rocky\/mosquitto\/" make -f Makefile binary<\/pre><\/p>\n\n\n\n
If all goes well we will have a result that looks like the below.
cc  -I. -I.. -I..\/include -I..\/..\/include -I..\/lib -DWITH_TLS -DWITH_TLS_PSK -DWITH_BRIDGE -DWITH_PERSISTENCE -DWITH_MEMORY_TRACKING -DWITH_SYS_TREE -DWITH_EC -DWITH_CONTROL -DWITH_UNIX_SOCKETS -DWITH_EPOLL -I..\/deps -I\/home\/rocky\/mosquitto\/ -DVERSION="\\"2.0.15\\"" -DWITH_BROKER -c websockets.c -o websockets.o\ncc  -I. -I.. -I..\/include -I..\/..\/include -I..\/lib -DWITH_TLS -DWITH_TLS_PSK -DWITH_BRIDGE -DWITH_PERSISTENCE -DWITH_MEMORY_TRACKING -DWITH_SYS_TREE -DWITH_EC -DWITH_CONTROL -DWITH_UNIX_SOCKETS -DWITH_EPOLL -I..\/deps -I\/home\/rocky\/mosquitto\/ -DVERSION="\\"2.0.15\\"" -DWITH_BROKER -c will_delay.c -o will_delay.o\ncc  -I. -I.. -I..\/include -I..\/..\/include -I..\/lib -DWITH_TLS -DWITH_TLS_PSK -DWITH_BRIDGE -DWITH_PERSISTENCE -DWITH_MEMORY_TRACKING -DWITH_SYS_TREE -DWITH_EC -DWITH_CONTROL -DWITH_UNIX_SOCKETS -DWITH_EPOLL -I..\/deps -I\/home\/rocky\/mosquitto\/ -DVERSION="\\"2.0.15\\"" -DWITH_BROKER -c ..\/lib\/will_mosq.c -o will_mosq.o\ncc  -I. -I.. -I..\/include -I..\/..\/include -I..\/lib -DWITH_TLS -DWITH_TLS_PSK -DWITH_BRIDGE -DWITH_PERSISTENCE -DWITH_MEMORY_TRACKING -DWITH_SYS_TREE -DWITH_EC -DWITH_CONTROL -DWITH_UNIX_SOCKETS -DWITH_EPOLL -I..\/deps -I\/home\/rocky\/mosquitto\/ -DVERSION="\\"2.0.15\\"" -DWITH_BROKER -c xtreport.c -o xtreport.o\ncc -L\/home\/rocky\/mosquitto\/cjson\/ -Wl,--dynamic-list=linker.syms mosquitto.o alias_mosq.o bridge.o bridge_topic.o conf.o conf_includedir.o context.o control.o database.o handle_auth.o handle_connack.o handle_connect.o handle_disconnect.o handle_ping.o handle_pubackcomp.o handle_publish.o handle_pubrec.o handle_pubrel.o handle_suback.o handle_subscribe.o handle_unsuback.o handle_unsubscribe.o keepalive.o logging.o loop.o memory_mosq.o memory_public.o misc_mosq.o mux.o mux_epoll.o mux_poll.o net.o net_mosq.o net_mosq_ocsp.o packet_datatypes.o packet_mosq.o password_mosq.o property_broker.o property_mosq.o persist_read.o persist_read_v234.o persist_read_v5.o persist_write.o persist_write_v5.o plugin.o plugin_public.o read_handle.o retain.o security.o security_default.o send_auth.o send_connack.o send_connect.o send_disconnect.o send_mosq.o send_publish.o send_suback.o send_subscribe.o send_unsuback.o send_unsubscribe.o service.o session_expiry.o signals.o strings_mosq.o subs.o sys_tree.o time_mosq.o topic_tok.o tls_mosq.o utf8_mosq.o util_mosq.o util_topic.o websockets.o will_delay.o will_mosq.o xtreport.o -o mosquitto  -ldl -lm -lrt -lssl -lcrypto \nmake[1]: Leaving directory '\/home\/rocky\/mosquitto\/mosquitto-2.0.15\/src'<\/pre><\/p>\n\n\n\n
Let’s now see if Mosquitto built correctly, let’s try to execute it.
src\/mosquitto -h<\/pre><\/p>\n\n\n\n
The following should be displayed, now we have a shiny new portable build of Mosquitto, we can run this in place now without an actual installation.
mosquitto version 2.0.15\n\nmosquitto is an MQTT v5.0\/v3.1.1\/v3.1 broker.\n\nUsage: mosquitto [-c config_file] [-d] [-h] [-p port]\n\n -c : specify the broker config file.\n -d : put the broker into the background after starting.\n -h : display this help.\n -p : start the broker listening on the specified port.\n      Not recommended in conjunction with the -c option.\n -v : verbose mode - enable all logging types. This overrides\n      any logging options given in the config file.\n\nSee https:\/\/mosquitto.org\/ for more information.<\/pre><\/p>\n\n\n\n
In the next post we will look at creating the configuration and SSL certificates used for authentication.<\/p>\n","protected":false},"excerpt":{"rendered":"
So far we have come up with the architecture and the database structure for the new IoT platform. Next we should address the MQTT communications portion. The MQTT protocol is proposed due to the publish\/subscribe features, lightweight overhead, and ability to encrypt\/secure the communications. It is possible to use HTTPS to post the data to the server but the overhead of headers and renegotiating the SSL encryption with new communication proves to be too much extra data to send. With HTTP posts the data also flows only one direction, from the sensor to the server. Reverse communication is not possible unless the connection is kept open. MQTT has the advantage […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[22,11],"tags":[],"class_list":["post-411","post","type-post","status-publish","format-standard","hentry","category-iot","category-linux"],"_links":{"self":[{"href":"https:\/\/www.drassal.net\/wp\/wp-json\/wp\/v2\/posts\/411","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.drassal.net\/wp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.drassal.net\/wp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.drassal.net\/wp\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.drassal.net\/wp\/wp-json\/wp\/v2\/comments?post=411"}],"version-history":[{"count":51,"href":"https:\/\/www.drassal.net\/wp\/wp-json\/wp\/v2\/posts\/411\/revisions"}],"predecessor-version":[{"id":462,"href":"https:\/\/www.drassal.net\/wp\/wp-json\/wp\/v2\/posts\/411\/revisions\/462"}],"wp:attachment":[{"href":"https:\/\/www.drassal.net\/wp\/wp-json\/wp\/v2\/media?parent=411"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.drassal.net\/wp\/wp-json\/wp\/v2\/categories?post=411"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.drassal.net\/wp\/wp-json\/wp\/v2\/tags?post=411"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}