1. Home
2. categories
3. dispatch
4. Project Curve Tracer - Serial Protocol Part 1

Looking for a Serial Protocol

As you know if you have been reading the other articles in this series, the end goal is to display the curve tracer data on a PC screen for the user to consume and to provide controls for the user to interact with the hardware.

I have looked around the interwebs for a serial protocol that is easy to use, lightweight, and has the ability to send multi-dimensional data, but I haven't found it. The closest think that I have found is the Telemetry library. This is an outstanding project in that it provides you with a serial protocol along with a graphical python script which can be used to plot the data coming out of the microcontroller.

Unfortunately, the library doesn't have the ability to send multi-dimensional data (x,y coordinates, for instance) nor the ability to send bulk data, such as an array.

So we will engage in that time-honored tradition of building up a serial protocol. There are hundreds - maybe thousands - out there, so the right one probably exists, but it might take us longer to find and customize it than to just write it and be done.

Goals

The current goals are:

• Data Integrity
• End-point delivery

At this time, there is no vision for ensuring content delivery (ACK, retransmit, etc.) at this level in the protocol. This means that the application layer will have to take care of ensuring content delivery, if necessary. It is possible that we will implement delivery assurance at a later time and have reserved a few bits so that we can take care of this aspect of transmission later.

The protocol described herein is a vision, not a final document. Issues brought about during implementation may drive some minor changes, although the spirit of the article should still be maintained.

Physical Medium

This library will be written using a UART with a focus on 8-bit point-to-point applications, but I see no reason that it wouldn't work on just about any point-to-point connection that is byte-aligned.

For the remainder of this article, we want to focus on the vision of the use model and won't get bogged down in the physical medium, but that is something that will need to be addressed at some point.

Publish/Subscribe Model

I must be honest, I am going to template my library as much as I can from Telemetry. I will just hack in those features that are required and I may end up with my own library out there on github. For now, it is just part of this project.

I wanted to start this series of posts with a quick vision. As I stated, I'm largely templating this from Telemetry, so much of this is available there. I will use a 'publish-subscribe' model in which two (or more on some types of buses) parties will have access to a serial channel and the data will be 'published' to a topic. Any parties that are interested in that topic will receive the data. If there are no subscribers, the data is simply sent, but not received.

Note that the picture shows all messages going through one channel and being automatically separated out on the other side. This is the basic idea of the publish/subscribe model that will be utilized. The image depicts one channel of publisher/subscription. In the UART, there are two channels, one for each direction. Each endpoint will have at least one publisher and at least one subscriber each in our application.

High-Level Publish-Subscribe Workflow

The library - above all - must be easy to port and easy to use! I want to focus on the usability part first simply because porting is a matter of coding it correctly. Usability has to come up front and doesn't just happen without some thought.

Publishing Data

The Simplest Example

The easiest thing to do with the pubserial library is to simply publish a string:

publish("foo", "my string");
//                  ^ string to be published, in quotes
//        ^ topic

The above code is somewhat equivalent to printf in that it will simply send the string specified to the particular topic.

The Simplest Numeric Example

The simplest possible publish would be to publish a single data point to a topic.

uint8_t bar = 1;
publish("foo,u8", &bar);
//                  ^ source of data
//            ^ format specifier
//        ^ topic

Note that there are three elements:s topic, format specifier, and data. The portion after the comma in the string is known as the 'format specifier' and will be explained in further detail later. If the data width is not specified, then the assumption is that the data in bar is a string.

The same thing could have been accomplished using an array notation for the variable declaration:

uint8_t bar[] = {1};
publish("foo,u8", bar);
//               ^source of data
//        ^topic

Which notation is utilized for single data points doesn't matter, but at a point later in the article, this notation will become the norm, so it helps to see it in both forms here on a single point of data.

Arrays

Next, we will likely want to post several related bits of data to a topic. We can do this using the colon ':' as the delimiter, similar to Python and JavaScript.

uint8_t bar[20] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19};
publish("foo:20,u8", bar);
//           ^ number of elements

Again, the width is not specified, so the assumption is that bar contains 8-bit data. The above would send 20 elements of the data contained within bar to all subscribers of the topic foo.

Format Specifier

Up to this point, we have been sending unsigned, 8-bit data. What if we want to send a signed 16-bit integer?

int8_t bar[] = {1};
publish("foo,s16", bar);
//            ^ format specifier

Now we have introduced the comma as a format specifier. Format specifiers can assign the data type to be sent. Valid values for the format specifier are listed in the table:

Multi-Dimensional Data

There are many times in which we wish to publish x and y at the same time for plotting or other purposes. It would be very nice to be able to do that with this model.

int16_t bar[] = {10};
uint16_t baz[] = {20};
publish("foo,s16,u16", bar, baz);
//                ^ second format specifier
//            ^ first format specifier

When multiple format specifiers are present in the topic string, then there should be multiple pieces of data to be sent with this transmission. In the above, a single datapoint is being sent that has two dimensions. You might think of this as a single data point in a scatter plot.

Note that strings cannot be sent with other data. If you need to send multiple strings, then that must occur as multiple publish sequences.

Putting it All Together

Finally, we get to the part where the real power of the library is flexed. We can use all of the above conventions together to send multi-dimensional data as an array for maximum transmission efficiency:

int16_t bar[100] = {10};
uint16_t baz[100] = {20};
publish("foo:100,s16,u16", bar, baz);

The above format would send the data contained in bar[0] through bar[99], baz[0] through baz[99] and publish the data to foo[0] through foo[99].

Admittedly, this isn't the simplest possible solution for a single application. The simplest solution would probably be to write a single function for your needs, maybe a publish16 and not have to worry about the rest. The extra complication of adding the string formatter is to add flexibility to the library. Using the string formatter allows the library to play well in a huge variety of applications and even allow run- time creation of the formatting string.

Publishing Limitations

The primary limitation is that string data and numeric data may not be mixed:

publish("foo,s16", "my string", &bar);
//            ^          ^ mixed string and numeric in one publish - invalid!

Additionally, RAM limitations may keep messages relatively small along with the number of dimensions. In our implementation, it seems reasonable to limit the dimensionality to '3' and the length to '128'.

Subscribing to Data

Again, assuming a C development environment, subscription should be as simple as specifying a topic and a function to execute when that topic is received.

/* write the function to execute when a topic is received */
void function myFooReceiver(PubSer* pubSerStruct){
    /* do something with the pubSerStruct data */
}

/* .... somewhere in the code .... */
subscribe("foo", &myFunction);

Note that the subscription specifies the topic ONLY. It does not specify data widths, array indices, or anything of the sort. All of this information will be conatined in the PubSer structure.

Additional Notes

The data model does not protect you from errors! For instance, if you specified that u16 data is to be sent, but specify that that it is to go into a s32 register in your subscriber call back, the misinterpretation will likely result in errors at run time, not at compile time.

Hardware Agnosticism

The library should also be able to utilize any byte-aligned hardware or communications channel as simply as possible.

Again, taking a cue from the Telemetry library, we would like to provide a function on which a library may be built. This will likely be provided in the form of a structure to initialize which points to the correct functions to execute for byte-aligned hardware access.

Use in the Curve Tracer Project

Once we get a nice little library going, we will be able to easily publish (x,y) data to a particular topic for our primary data transfer:

publish("curvetrace:128,s16,s16", volts, amps);

Additionally, the microcontroller will be able to subscribe to feeds such as 'frequency' and 'waveshape', making modification of the operating characteristics of the software trivial.

In part 2, we will translate all of the above into a formatted stream of bytes so that we can think about sending data in a byte-aligned chunk across an interface!

• Previous Article
• Next Article

• Previous in dispatch
• Next in dispatch