MQTT Adaptor
Introduction
MQTT is a machine-to-machine (M2M)/"Internet of Things" connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium.
MQTT adaptor implements on paho.mqtt.golang and helps to communicate with the MQTT broker to interact with linking equipment.
Gossip
Data Structure
As we know, MQTT is structure-agnostic, so there is no standard topic naming schema and payload format. The way the publisher organizes the data structure will directly affect the usage of the subscriber. In the community, we have summarized two common patterns. Let's take a look below.
The first pattern can be named as Attributed Topic: the publisher flattens properties into topics, and then sends the property's payload to corresponding topic. It has a representative on Github: the Homie MQTT convention.
Homie is interesting, its biggest feature is self-discovery, which means subscriber doesn't need to know the data structure and just subscribe the root topic, and then the convention implementation client will reflect all properties, including the name, description, value, type and so on. However, Attributed Topic pattern creates a lot of topics, so it needs a Homie-like convention to ensure standardization and extensibility.
Another pattern of directly compressing attributes into one payload can be named as Attributed Message: The publisher serializes properties as one target format, such as XML, JSON or custom form, and then sends the entire serialized result to a topic.
Attributed Message pattern is topic saving, but the subscriber needs to know how to deserialize the payload in each topic and understand the organizational structure of data. A better way is to use the same serialization format in all topics and introduce a hierarchical description of the data structure. For example, if publisher choose JSON as the serialization format, publisher can attach the JSONSchema of data structure in another topic.
Operation
In MQTT, there are only two ways of pub/sub for data: either perform pub/sub on the same topic, or divide pub/sub into two topics.
The first way is not popular, it may need to add operation commands to the payload.
Although a system that uses declarative management(like Kubernetes) can avoid the above imperative operation, it is necessary to introduce a sub when the publisher did pub, which is unacceptable in an extremely low-power environment.
Therefore, the second way will be more easily to accept. Since the properties have been flatten in Attributed Topic pattern, the publisher can send the data to the topic with special suffix corresponding to the property. For example, the Homie prefers to use a topic with set
ending to receive value changes.
The same is true for Attributed Message pattern, expect that the publisher need to choose whether to only send modified properties or all properties.
Convention
MQTTDevice integrated the configuration of MQTT extension.
AttributedTopic Pattern
Specifies pattern: AttributedTopic
to interact with equipment that flattens its properties in multiple topics.
Specifies templated topic with :path
keyword to render the target topic corresponding to property name.
Or explicitly indicates the :path
with path
field.
Specifies a writable property with readOnly: false
.
Changes the writable property to publish to another topic.
Note that :operator
can be overwritten.
AttributedMessage
Specifiy pattern: AttributedMessage
to interact with equipment that compresses its properties in one topic.
note
AttributedMessage
pattern only supports JSON format payload content at present.
If the JSON of payload content looks as below:
Extracts the content via property name
.
Or indicate the retrival JSONPath in path
field.
Specifies a writable property with readOnly: false
.
Changes the publish topic.
JSONPath
note
JSONPath is only available in AttributedMessage
pattern.
MQTT adaptor has integrated both tidwall/gjson and tidwall/sjson.
For Read Only property, path
field can accept GJSON Path Syntax, which is an amazing and richable path retrieval mechanism.
However, against Writable property, path
field can only accept restricted SJSON Path Syntax.
In order to ensure consistent read/write path for a property, MQTT adaptor prevents the following paths on Writable property:
children.-1
is not allowed;children|@reverse
is not allowed;child*.2
is not allowed;c?ildren.0
is not allowed;friends.#.first
is not allowed.
User Story
Imagine that our home appliances are very smart and can actively report their status information to an MQTT broker, and then we will use MQTTDevice
to link these information. For example, our kitchen door can tell us its production information, its closed status and so on.
We can define a MQTTDevice
device link with AttributedTopic
pattern to watch our kitchen door.
In
AttributedTopic
pattern, eachproperty
is a topic. By default, thename
of property can be used as the:path
keyword to render the topic, and finally get the corresponding topic to subscribe.
The kitchen light also reports its properties to MQTT broker and allow us to control it remotely.
We can use the writable properties of MQTTDevice
device link to control the kitchen light.
Use
readOnly: false
to conifgure a writable property. In addition, the property leveloperator
can override the definition of the protocol level inAttributedTopic
pattern.
For example, we can turn on the kitchen light and adjust it to mid level.
Futher, we can monitor the status of door and light in the same MQTTDevice
device link.
Recently we bought a new smart bedroom light, but we found that the format of the transmitted data is different from the previous one.
We can define a MQTTDevice
device link with AttributedMessage
pattern to monitor the new bedroom light.
In
AttributedMessage
pattern, the whole link is a topic. By default, thename
of property can be used as the retrieval path of payload content.
We can modify the above MQTTDevice
device link if needed to turn off the new bedroom light.
Registration Information
Versions | Register Name | Endpoint Socket | Available |
---|---|---|---|
v1alpha1 | adaptors.edge.cattle.io/mqtt | mqtt.sock | * |
Support Model
Kind | Group | Version | Available |
---|---|---|---|
MQTTDevice | devices.edge.cattle.io | v1alpha1 | * |
Support Platform
OS | Arch |
---|---|
linux | amd64 |
linux | arm |
linux | arm64 |
Usage
Authority
Grant permissions to Octopus as below :
Example
Specifies a
MQTTDevice
device link to subscribe the information of kitchen room door.apiVersion: edge.cattle.io/v1alpha1kind: DeviceLinkmetadata:name: kitchen-doorspec:adaptor:node: edge-workername: adaptors.edge.cattle.io/mqttmodel:apiVersion: "devices.edge.cattle.io/v1alpha1"kind: "MQTTDevice"template:metadata:labels:device: kitchen-doorspec:protocol:pattern: "AttributedTopic"client:server: "tcp://test.mosquitto.org:1883"message:topic: "cattle.io/octopus/home/status/kitchen/door/:path"properties:- name: "state"path: "state"description: "The state of door"type: "string"annotations:type: "enum"format: "open,close"- name: "width"path: "width"description: "The width of door"type: "float"annotations:unit: "meter"- name: "height"path: "height"description: "The height of door"type: "float"annotations:unit: "meter"- name: "material"path: "material"description: "The material of light"type: "string"Specifies a
MQTTDevice
device link to manage the bedroom light.apiVersion: edge.cattle.io/v1alpha1kind: DeviceLinkmetadata:name: bedroom-lightspec:adaptor:node: edge-workername: adaptors.edge.cattle.io/mqttmodel:apiVersion: "devices.edge.cattle.io/v1alpha1"kind: "MQTTDevice"template:metadata:labels:device: bedroom-lightspec:protocol:pattern: "AttributedMessage"client:server: "tcp://test.mosquitto.org:1883"message:topic: "cattle.io/octopus/home/bedroom/light/:operator"operator:write: "set"properties:- name: "switch"path: "switch"description: "The switch of light"type: "boolean"readOnly: false- name: "gear"path: "action.gear"description: "The gear of light"type: "string"readOnly: falseannotations:type: "enum"format: "low,mid,high"- name: "power"path: "parameter.power"description: "The power of light"type: "float"annotations:group: "parameter"unit: "watt"- name: "luminance"path: "parameter.luminance"description: "The luminance of light"type: "int"annotations:group: "parameter"unit: "luminance"- name: "manufacturer"path: "production.manufacturer"description: "The manufacturer of light"type: "string"annotations:group: "production"- name: "productionDate"path: "production.date"description: "The production date of light"type: "string"annotations:group: "production"type: "datetime"standard: "ISO 8601"format: "YYYY-MM-DDThh:mm:ss.SSZ"- name: "serviceLife"path: "production.serviceLife"description: "The service life of light"type: "string"annotations:group: "production"type: "duration"standard: "ISO 8601"format: "PYYMMDD"
For more MQTTDevice
device link examples, please refer to the deploy/e2e directory and make a quick experience with deploy/e2e/simulator.yaml.
MQTTDevice
Parameter | Description | Schema | Required |
---|---|---|---|
metadata | metav1.ObjectMeta | false | |
spec | Defines the desired state of MQTTDevice . | MQTTDeviceSpec | true |
status | Defines the observed state of MQTTDevice . | MQTTDeviceStatus | false |
MQTTDeviceSpec
Parameter | Description | Schema | Required |
---|---|---|---|
protocol | Specifies the protocol for accessing the MQTT service. | MQTTDeviceProtocol | true |
properties | Specifies the properties of device. | []MQTTDeviceProperty | false |
MQTTDeviceStatus
Parameter | Description | Schema | Required |
---|---|---|---|
properties | Reports the properties of device. | []MQTTDeviceStatusProperty | false |
MQTTDeviceProtocol
Parameter | Description | Schema | Required |
---|---|---|---|
pattern | Specifies the pattern of MQTTDevice protocol. | MQTTDevicePattern | true |
client | Specifies the client settings. | MQTTClientOptions | true |
message | Specifies the message settings. | MQTTMessageOptions | true |
MQTTDevicePattern
Parameter | Description | Schema |
---|---|---|
AttributedMessage | Compress properties into one message, one topic has its all property values. | string |
AttributedTopic | Flatten properties to topic, each topic has its own property value. | string |
MQTTDeviceProperty
Parameter | Description | Schema | Required |
---|---|---|---|
annotations | Specifies the annotations of property. | map[string]string | false |
name | Specifies the name of property. | string | true |
description | Specifies the description of property. | string | false |
readOnly | Specifies if the property is read-only, default to true . | *bool | false |
type | Specifies the type of property. | MQTTDevicePropertyType | true |
value | Specifies the value of property, only available in the writable property. | MQTTDevicePropertyValue | false |
path | Specifies the path for rendering the :path keyword of topic, default is the same as the name . In AttributedTopic pattern, this path will be rendered on topic; In AttributedMessage pattern, this path should be a JSONPath which can access the payload content. | string | false |
operator | Specifies the operator for rendering the :operator keyword of topic. | *MQTTMessageTopicOperator | false |
qos | Specifies the QoS of the message, only available in AttributedTopic pattern. The default value is 1 . | *MQTTMessageQoSLevel | false |
retained | Specifies if the last published message to be retained, only available in AttributedTopic pattern. The default is true . | *bool | false |
In fact, MQTT adaptor will return the original data received by MQTT broker. Therefore, the meaning of
type
is not to tell MQTT adaptor how to deal with the payload, but for user to describe what is expected.
MQTTDeviceStatusProperty
Parameter | Description | Schema | Required |
---|---|---|---|
annotations | Reports the annotations of property. | map[string]string | false |
name | Reports the name of property. | string | false |
description | Reports the description of property. | string | false |
readOnly | Reports if the property is read-only. | *bool | false |
type | Reports the type of property. | MQTTDevicePropertyType | false |
value | Reports the value of property. | MQTTDevicePropertyValue | false |
path | Reports the path for rendering the :path keyword of topic, default is the same as the name | string | false |
operator | Reports the operator for rendering the :operator keyword of topic. | *MQTTMessageTopicOperator | false |
qos | Reports the QoS of the message. | *MQTTMessageQoSLevel | false |
retained | Reports if the last published message to be retained. | *bool | false |
updatedAt | Reports the updated timestamp of property. | *metav1.Time | false |
MQTTDevicePropertyType
Parameter | Description | Schema |
---|---|---|
string | Property data type is string. | string |
int | Property data type is int. | string |
float | Property data type is float. | string |
boolean | Property data type is boolean. | string |
array | Property data type is array. | string |
object | Property data type is object. | string |
MQTTDevicePropertyValue
MQTTDevicePropertyValue needs to be inputed with the corresponding content according to the type
. For example: