GEX is designed to be modular and easy to extend. It is composed of a set of functional blocks (also called \textit{units}), sometimes available in more than one instance, which can be configured by the user to fit their application needs. The firmware is built around a \textit{core framework} which provides services to the functional blocks, such as a settings storage, resource allocation, message delivery and periodic updates.
The firmware is built around a \textit{core framework} which provides services to units, such as the settings storage, resource allocation, message delivery, and periodic updates. In this chapter, we will focus on the structure of this framework and the services provided by it.
In this chapter we will focus on the general function of the GEX module and will look at the services provided by the core framework. Individual functional blocks and the communication protocol will be described in \cref{sec:tinyframe,sec:units-overview}.
\iffalse
This and the following parts were written after implementing and evaluating the first hardware prototype and its firmware, therefore rather than describing the development process, it tends to talk about the completed solution and the decisions taken.
\fi
\section{User's View of GEX}
Before going into implementation details, we'll have a look at GEX from the outside, how the end user will see it. This should give the reader some context to better orient themselves in the following sections and chapters investigating the internal structure of the firmware and the communication protocol.
\begin{figure}[h]
\centering
\includegraphics[scale=.95]{img/users-view.pdf}
\caption{\label{fig:users-view-of-gex}Physical user interface of a GEX module}
\end{figure}
The GEX firmware can be flashed to a STM32 Nucleo or Discovery board or a custom \gls{PCB}. Discovery boards are equipped with a ``user \gls{USB}'' connector routed to the application \gls{MCU}; Nucleo boards have only the ST-Link \gls{USB} connector and, since the ST-Link version 2.1, offer a built-in USB-serial converter leading to one of the \gls{MCU}'s hardware \glspl{UART}.
After powering on, GEX loads its configuration from the Flash memory, configures its peripherals, sets up the function blocks and enables the selected communication interface(s). From this point, the module is operational and can be configured or used as needed by the user.
The physical user interface of the module is shown in \cref{fig:users-view-of-gex}. When a \gls{USB} cable connects the board to a \gls{PC}, the \gls{PC}\gls{OS} enumerates it and either recognizes the communication interface as \gls{CDCACM} (Virtual serial port), or leaves it without a software driver attached, to be accessed directly as raw \gls{USB} endpoints, based on GEX settings.
The connection using a physical UART and a USB/UART adapter works in a very similar way, as does the wireless connection (described in more detail below).
\subsection{Updating GEX Settings}
Two ways exist in which the module's settings can be modified: via the virtual Mass Storage disk, and through the control interface. We'll look at the file system method here; the API for a programmatic loading and updating of configuration will be explained in \cref{sec:tf-bulk-rw}.
The board is equipped with a button or a jumper labeled Lock. When the button is pressed or the jumper removed (or inserted, the polarity is configured in the firmware), the Mass Storage \gls{USB} interface is enabled. For the user, this means that a new disk will appear on their computer which they can open in a file manager. The disk provides a read/write access to configuration files.
The user edits a file as needed and saves it back to the disk. GEX processes the new content, tries to apply the changes and generates an updated version that includes any error messages or newly generated sections (when a new unit was registered). For the \gls{PC}\gls{OS} to recognize this change, the Mass Storage device momentarily reports that the media is unavailable to force the \gls{OS} to reload it. This is a similar mechanism to what happens when a memory card is removed from a reader. Now the user can reload the file in their editor, inspect the updated content and correct any possible mistakes. The settings, when applied successfully, should be immediately available to test using the communication interface. When everything is to the user's satisfaction, the updated settings are committed to the device's Flash memory by pressing the Lock button again, or replacing the jumper.
\subsection{Connecting Using the Wireless Module}
In the case when a wireless module is installed on the \gls{PCB} and GEX is configured to use it, the radio link becomes a fallback connection when the \gls{USB} peripheral does not get enumerated within a short time after start-up.
To use it, the user needs to connect a wireless gateway module to their host \gls{PC} and use the radio link instead of a \gls{USB} cable. This connection works in a way similar to the hardware UART interface: it can be used to read and modify the configuration files and to access the functional blocks; the difference lies in a slightly different protocol required to communicate with the gateway itself, e.g., to pair it with the GEX module.
\subsection{Using the Control Interface}
Now that GEX is connected and configured, the user can start using it. This involves writing a program in C or Python that uses the GEX client library, using the Python library from MATLAB, or controlling GEX using a \gls{GUI} front-end built on those libraries. The configuration can be stored in the module, but it is also possible to temporarily replace it using the control \gls{API}. This way, the settings are loaded automatically when the user's program starts and may differ for different programs.
\section{Internal Structure Block Diagram}
\section{Internal Structure Block Diagram}
The data flows and other internal logic of the firmware are depicted in \cref{fig:gex-internal}, with more explanation following in this chapter. The interchangeable role of the three communication interfaces can be clearly seen in the diagram, as well as the central role of the message queue which decouples interrupts from the processing thread.
The data flows and other internal logic of the firmware are depicted in \cref{fig:gex-internal}, with more explanation following in this chapter. The interchangeable role of the three communication interfaces can be clearly seen in the diagram, as well as the central role of the message queue, which decouples interrupts from the processing thread.
\begin{figure}[h]
\begin{figure}[h]
\centering
\centering
@ -53,25 +12,40 @@ The data flows and other internal logic of the firmware are depicted in \cref{fi
\caption{\label{fig:gex-internal}Block diagram showing the internal logic in the GEX firmware}
\caption{\label{fig:gex-internal}Block diagram showing the internal logic in the GEX firmware}
\end{figure}
\end{figure}
The core framework forms the skeleton of the firmware and usually does not need any changes when new user-facing features are added. When the firmware is ported to a different STM32 microcontroller, the framework is not difficult to adjust and the whole process can be accomplished in a few hours. The porting of units to a different platforms is significantly more challenging.
\noindent
The framework provides the following services to units:
The firmware is built on FreeRTOS (\cref{sec:freertos}) and a number of its synchronization objects and patterns are used to make its operation more robust.
The firmware is built around FreeRTOS (\cref{sec:freertos}) and a number of its synchronization objects and patterns are used to make its operation more robust.
\subsection{Message and Job Queue}
\subsection{Message and Job Queue}
The message and job queue, seen in \cref{fig:gex-internal}, is used to decouple asynchronous interrupts from message transmission. All three communication interfaces use interrupts for the asynchronous handling of incoming messages. The same interrupt handler receives an event after a transmission was completed. The queue ensures that messages can be received during the transmission of a large response that demands the use of multiple transmissions.
The message and job queue, seen in \cref{fig:gex-internal}, is used to decouple asynchronous interrupts from message transmission. All three communication interfaces use interrupts for the asynchronous handling of incoming messages. The same interrupt handler receives an event after a transmission was completed. The queue ensures that messages can be received during the transmission of a large response that demands the use of multiple transmissions.
The ``transmission complete'' interrupt signals this fact to the message processing task using a binary semaphore. The semaphore is released in the interrupt and take before a new block of data is transmitted. If more data needs to be transmitted, the queue task waits on the semaphore and enters a Blocked state until the semaphore becomes available again.
The ``transmission complete'' interrupt signals this fact to the message processing task using a binary semaphore. The semaphore is released in the \gls{ISR} and taken when a new block of data is to be transmitted. If more data needs to be transmitted, the queue task waits for the semaphore and enters a Blocked state until the semaphore becomes available again.
Two mutexes are used in the firmware: one that guards access to TinyFrame until the previous message was fully transmitted, and one to guard a shared memory buffer used, among other, by unit drivers during the serialization and parsing of a configuration file. The hardware resource registry (explained in \cref{sec:res-allocation}) does not need mutexes for individual resources, as a concurrent access to those fields can never happen thanks to the way the system is organized.
Two mutexes are used in the firmware: one that guards access to TinyFrame until the previous message was fully transmitted, and one to guard a shared memory buffer (reused in several different places to save memory and avoid its re-allocation). The hardware resource registry (explained in \cref{sec:res-allocation}) does not need mutexes for individual resources, as concurrent access to those fields can never happen; resources are always taken or released sequentially by the same task.
\section{Unit Life Cycle and Internal Structure}\label{sec:units-function}
GEX's user-facing functions are implemented in \textit{unit drivers}. Those are mutually independent modules in the firmware that the user can enable and configure using a configuration file. There can be multiple instances of each unit type. However, we are limited by hardware constraints: for example, there may be only one \gls{ADC} peripheral, two \gls{SPI} ports and so on. The assignment of those hardware resources to units is handled by the \textit{resource registry} (\cref{sec:res-allocation}).
GEX's user-facing functions, units, are implemented in \textit{unit drivers}. Those are independent modules in the firmware that the user can enable and configure, in one or more instances. In practice, we are limited by hardware constraints: i.e., there may be only one \gls{ADC} peripheral, or two \gls{SPI} ports. The assignment of those hardware resources to units is handled by the \textit{resource registry} (\cref{sec:res-allocation}).
Each unit is defined by a section in the configuration file \verb|UNITS.INI|. It is given a name and a \textit{callsign}, which is a number that serves as an address for message delivery. A unit is internally represented by a data object with the following structure:
\begin{itemize}
Each unit is identified by a name and a \textit{callsign}, which is a number that serves as an address for message delivery. A unit is internally stored as a data object with the following structure:
\begin{itemize}[itemsep=0pt]
\item Name
\item Name
\item Callsign (one byte)
\item Callsign (one byte)
\item Configuration parameters loaded from the unit settings
\item Configuration parameters loaded from the unit settings
@ -79,68 +53,11 @@ Each unit is defined by a section in the configuration file \verb|UNITS.INI|. It
\item A reference to the unit driver
\item A reference to the unit driver
\end{itemize}
\end{itemize}
The unit driver handles commands sent from the host \gls{PC}, initializes and de-initializes the unit based on its settings, and implements other aspects of the unit's function, such as periodic updates and interrupt handling. Unit drivers may expose public \gls{API} functions to make it possible to control the unit from a different driver, allowing the creation of ``macro units''.
The unit driver handles commands sent from the host \gls{PC}, initializes and de-initializes the unit based on its settings, and implements other aspects of its function, such as periodic updates and interrupt handling.
\section{Source Code Layout}
\begin{wrapfigure}[22]{r}{0.4\textwidth}
\scriptsize\vspace{-3em}
\begin{verbatim}
├── build
│ ├── firmware.bin
│ └── firmware.dfu
├── Drivers
│ ├── CMSIS
│ │ └── Device / ST / STM32F0xx
│ └── STM32F0xx_HAL_Driver
├── Middlewares / Third_Party / FreeRTOS
├── Src
│ └── main.c
├── User
│ ├── USB / STM32_USB_Device_Library
│ │ ├── Class
│ │ │ ├── CDC
│ │ │ ├── MSC
│ │ │ └── MSC_CDC
│ │ └── Core
│ ├── platform
│ │ ├── plat_compat.h
│ │ └── platform.c
│ ├── units
│ │ ├── adc
│ │ ├── digital_out
│ │ ...
│ ├── FreeRTOSConfig.h
│ └── gex.mk
└── Makefile
\end{verbatim}
\vspace{-1em}
\caption{\label{fig:repo-structure} The general structure of the source code repository}
\end{wrapfigure}
Looking at the source code repository (\cref{fig:repo-structure}), at the root we'll find the device specific driver libraries and support files provided by ST Microelectronics, the FreeRTOS middleware, and a folder called \verb|User| containing the GEX application code. This division is useful when porting the firmware to a different microcontroller, as the GEX folder is mostly platform-independent and can be simply copied (of course, adjustments are needed to accompany different hardware peripheral versions etc.). The GEX core framework consists of everything in the \verb|User| folder, excluding the \verb|units| directory in which the individual units are implemented. Each unit driver must be registered in the file \verb|platform.c| to be available for the user to select. The file \verb|plat_compat.h| includes platform-specific headers and macros, defining parameters such as pin assignments or the clock speed.
The \gls{USB} Device library, which had to be modified to support a composite class, is stored inside the \verb|User| folder too, as it is compatible with all STM32 microcontrollers that support \gls{USB}.
\section{Functions of the Core Framework}
The core framework forms the skeleton of the firmware and usually does not need any changes when new user-facing features are added. It provides the following services:
\item The communication port with different back-ends: \gls{USB}, \gls{UART}, wireless (\cref{sec:com-ports})
\item Message sending and delivery (\cref{sec:message_passing})
\item Interrupt management and routing to functional blocks (\cref{sec:irq-routing})
\item Virtual mass storage for configuration files (explained in \cref{sec:fat16})
\end{itemize}
When the firmware needs to be ported to a different STM32 microcontroller, the core framework is relatively straightforward to adapt and the whole process can be accomplished in a few hours. The time consuming part is modifying the functional blocks to work correctly with the new device's hardware.
When the units configuration file is modified, all units are de-initialized and removed. The binary settings are then updated based on the new values, verifying that the requested resources are available, and the units that can be enabled are subsequently initialized and made available to the user.
@ -148,18 +65,18 @@ When the firmware needs to be ported to a different STM32 microcontroller, the c
\caption{\label{fig:resource-repository}An example allocation in the resource registry}
\caption{\label{fig:resource-repository}An example allocation in the resource registry}
\end{figure}
\end{figure}
The microcontroller provides a number of hardware resources that require exclusive access: GPIO pins, peripheral blocks (\gls{SPI}, \gls{I2C}, \gls{UART}\textellipsis), \gls{DMA} channels. If two units tried to control the same pin, the results would be unpredictable; similarly, with a multiple access to a serial port, the output would be a mix of the data streams and completely useless.
The microcontroller provides a number of hardware resources that require exclusive access: GPIO pins, peripheral blocks (\gls{SPI}, \gls{I2C}, \gls{UART}\textellipsis), and \gls{DMA} channels. If two units tried to control the same pin, the results would be unpredictable; similarly, the output of a multiply-accessed serial port could become a useless mix of the different data streams.
To prevent a multiple access, the firmware includes a \textit{resource registry} (\cref{fig:resource-repository}). Each individual resource is represented by a field in a resource table together with its owner's callsign. Initially all resources are free, except for those not available on the particular platform (e.g., a \gls{GPIO} pin PD1 may be disabled if not present on the microcontroller's package).
To prevent multiple access, the firmware includes a \textit{resource registry} (\cref{fig:resource-repository}). Each individual resource is represented by a field in a resource table together with its owner's callsign. Initially all resources are free, except for those not available on the particular platform (e.g., a \gls{GPIO} pin PD1 may be disabled if not present on the microcontroller's package).
The resources used by the core framework are taken by a virtual unit \verb|SYSTEM| on start-up to prevent conflicts with the user's units. This is the case of the status \gls{LED}, the Lock button, \gls{USB} pins, the communication \gls{UART}, the pins and an \gls{SPI} peripheral connecting the wireless module, pins used for the crystal oscillator, and the timer/counter which provides the system timebase.
The resources used by the core framework are taken by a virtual unit \verb|SYSTEM| on start-up to prevent conflicts with the user's units. This is the case of the status \gls{LED}, the Lock button, \gls{USB} pins, the communication \gls{UART}, the pins and an \gls{SPI} peripheral connecting the wireless module, pins used for the crystal oscillator, and the timer/counter which provides the system timebase.
\caption{\label{fig:settings-storage}Structure of the settings subsystem}
\caption{\label{fig:settings-storage}Structure of the settings subsystem}
\end{figure}
\end{figure}
@ -167,44 +84,29 @@ The system and unit settings are written, in a binary form, into designated page
As the settings persist after a firmware update, it is important to maintain backward compatibility. This is achieved by prefixing the settings block of each unit with a version number. When the settings are loaded by a new version of the firmware, it first checks the version and decides whether to use the old or new format. When the settings are next changed, the new format will be used.
As the settings persist after a firmware update, it is important to maintain backward compatibility. This is achieved by prefixing the settings block of each unit with a version number. When the settings are loaded by a new version of the firmware, it first checks the version and decides whether to use the old or new format. When the settings are next changed, the new format will be used.
The INI files, which can be edited through the communication \gls{API} or using a text editor with the virtual mass storage, are parsed and generated on demand and are never stored in the Flash or \gls{RAM}, other than in short temporary buffers. The INI parser processes the byte stream on-the-fly as it is received, and a similar method is used to build a INI file from the configured units and system settings.
\iffalse
The INI files, which can be edited through the communication \gls{API} or using a text editor through the virtual mass storage, are parsed and generated on demand and are never stored in the Flash or \gls{RAM}, other than in short temporary buffers. The INI parser processes the byte stream on-the-fly as it is received, and a similar method is used to build a INI file from the configured units and system settings. \fi% said elsewhere
The firmware supports three different communication ports: hardware \gls{UART}, \gls{USB} (virtual serial port), and a wireless connection. Each interface is configured and accessed in a different way, but for the rest of the firmware (and for the \gls{PC}-side application) they all appear as a full duplex serial port. To use interfaces other than \gls{USB}, the user must configure those in the system settings (a file \verb|SYSTEM.INI| on the configuration disk).
At start-up, the firmware enables the \gls{USB} peripheral, configures the device library and waits for enumeration by the host \gls{PC}. When not enumerated, it concludes the \gls{USB} cable is not connected, and tries some other interface. The \gls{UART} interface cannot be tested as reliably, but it is possible to measure the voltage on the Rx pin. When idle, a \gls{UART} Rx line should be high (here 3.3\,V). The wireless module, when connected using \gls{SPI}, can be detected by reading a register with a known value and comparing those.
GEX uses vid:pid \verb|1209:4c60| and the wireless gateway \verb|1209:4c61|. The \gls{USB} interface uses the \gls{CDCACM}\gls{USB} class (\cref{sec:cdc-acm}) and consists of two bulk endpoints with a payload size of up to 64 bytes.
One of the key functions of the core framework is to deliver messages from the host \gls{PC} to the right units.
\subsubsection{Communication UART}
Two groups of messages exist: \textit{system messages} and \textit{unit messages}. System messages can, for instance, access the INI files, or request a list of available units. Unit messages are addressed to a particular unit, and their payload format is defined by the unit's driver. An incoming message is inspected and delivered to the appropriate recipient, or responded to with an error message.
The parameters of the communication \gls{UART} (such as the baud rate) are defined in \verb|SYSTEM.INI|. It is mapped to pins PA2 and PA3; this is useful with STM32 Nucleo boards that do not include a User \gls{USB} connector, but provide a \gls{USB}-serial bridge using the on-board ST-Link programmer, connected to those pins.
In addition to message delivery, the core framework also provides event reporting. Events are messages generated by units and sent to the host to notify it about asynchronous events.
This is identical to the \gls{USB} connection from the \gls{PC} application's side, except a physical \gls{UART} is necessarily slower and does not natively support flow control. The use of the Xon and Xoff software flow control is not practical with binary messages that could include those bytes by accident, and the ST-Link \gls{USB}-serial adapter does not implement hadware flow control.
This high-level functionality resides above the framing protocol, which will be described in \cref{sec:tinyframe}. The message format is shown in \cref{sec:unit_requests_reports}.
\subsubsection{Wireless Connection}
The wireless connection uses an on-board communication module and a separate device, a wireless gateway, that connects to the \gls{PC}. The wireless gateway is interfaced differently from the GEX board itself, but it also shows as a virtual serial port on the host \gls{PC}. This is required to allow communicating with the gateway itself through the \gls{CDCACM} interface in addition to addressing the end devices.
One of the key functions of the core framework is to deliver messages from the host \gls{PC} to the right units. This functionality resides above the framing protocol, which will be described in \cref{sec:tinyframe}.
A message that is not a response in a multi-part session (this is handled by the framing library) is identified by its Type field. Two main groups of messages exist: \textit{system messages} and \textit{unit messages}. System messages can access the INI files, query a list of the available units, restart the module etc. Unit messages are addressed to a particular unit by their callsign (see \cref{sec:units-function}), and their payload format is defined by the unit driver. The framework reads the message type, then the callsign byte, and tries to find a matching unit in the unit list. If no unit with the callsign is found, an error response is sent back, otherwise the unit driver is given the message to handle it as required.
The framework provides one more messaging service to the units: event reporting. An asynchronous event, such as an external interrupt, an \gls{ADC} trigger or an \gls{UART} data reception needs to be reported to the host. This message is annotated by the unit callsign so the user application knows its origin.
Interrupts are an important part of almost any embedded application. They provide a way to rapidly react to asynchronous external or internal events, temporarily leaving the main program, jumping to an interrupt handler routine, and then returning back after the event is handled. Interrupts are also the way FreeRTOS implements multitasking without a multi-core processor.
In \arm Cortex-M0 the interrupt handlers table, defining which routine is called for which interrupt, is stored in the program memory and cannot be changed at run-time. This is a complication for the modular structure of GEX where different unit drivers may use the same peripheral, and we would want to dynamically assign the interrupt handlers based on the active configuration.
Interrupts are an important part of almost any embedded application. They provide a way to rapidly react to asynchronous external or internal events, temporarily leaving the main program, jumping to an interrupt handler routine, and then returning back after the event is handled. Interrupts are also the way FreeRTOS implements multitasking without a multi-core processor.
In the \arm Cortex-M0-based STM32F072, used in the initial GEX prototypes, the interrupt handlers table, defining which routine is called for which interrupt, is stored in the program memory and cannot be changed at run-time. This is a complication for the modular structure of GEX where different unit drivers may use the same peripheral, and we would want to dynamically assign the interrupt handlers based on the active configuration. Let's have a look at an interrupt handler, in this case handling four different \gls{DMA} channels, as is common in STM32 microcontrollers:
Let us have a look at a sample interrupt handler, in this case serving four different \gls{DMA} channels, as is common in STM32 microcontrollers:
It is evident that multiple units might need to use the same interrupt handler, even at the same time, since each \gls{DMA} channel is configured, and works, independently. GEX implements a redirection scheme to accomplish such interrupt sharing: All interrupt handlers are defined in one place, accompanied by a table of function pointers. When a unit driver wants to register an interrupt handler, it stores a pointer to it in this redirection table. Then, once an interrupt is invoked, the common handler checks the corresponding entry in the table and calls the referenced routine, if any. Conversely, when a unit driver de-initializes a unit, it removes all interrupt handlers it used, freeing the redirection table slots for other use.
It is evident that multiple units might need to use the same handler, even at the same time, since each \gls{DMA} channel is configured, and works, independently. GEX implements a redirection scheme to accomplish such interrupt sharing: All interrupt handlers are defined in one place, accompanied by a table of function pointers. When a unit driver wants to register an interrupt handler, it stores a pointer to it in this redirection table. Then, once an interrupt is invoked, the common handler checks the corresponding entry in the table and calls the referenced routine, if any. Conversely, when a unit driver de-initializes a unit, it removes all interrupt handlers it used, freeing the redirection table slots for other use.
\section{Source Code Layout}
\begin{wrapfigure}[22]{r}{0.4\textwidth}
\scriptsize\vspace{-3em}
\begin{verbatim}
├── build
│ ├── firmware.bin
│ └── firmware.dfu
├── Drivers
│ ├── CMSIS
│ │ └── Device / ST / STM32F0xx
│ └── STM32F0xx_HAL_Driver
├── Middlewares / Third_Party / FreeRTOS
├── Src
│ └── main.c
├── User
│ ├── USB / STM32_USB_Device_Library
│ │ ├── Class
│ │ │ ├── CDC
│ │ │ ├── MSC
│ │ │ └── MSC_CDC
│ │ └── Core
│ ├── platform
│ │ ├── plat_compat.h
│ │ └── platform.c
│ ├── units
│ │ ├── adc
│ │ ├── digital_out
│ │ ...
│ ├── FreeRTOSConfig.h
│ └── gex.mk
└── Makefile
\end{verbatim}
\vspace{-1em}
\caption{\label{fig:repo-structure} The general structure of the source code repository}
\end{wrapfigure}
Looking at the GEX source code repository (\cref{fig:repo-structure}), at the root we'll find the device specific driver libraries and support files provided by ST Microelectronics, the FreeRTOS middleware, and a folder called \verb|User| containing the GEX application code. This division is useful when porting the firmware to a different microcontroller, as the GEX folder is mostly platform-independent and can be simply copied (of course, adjustments are needed to accompany different hardware peripheral versions etc.). The GEX core framework consists of everything in the \verb|User| folder, excluding the \verb|units| directory in which the individual units are implemented. Each unit driver must be registered in the file \verb|platform.c| to be available for the user to select. The file \verb|plat_compat.h| includes platform-specific headers and macros, defining parameters such as pin assignments or the clock speed.
The \gls{USB} Device library, which had to be modified to support a composite class, is stored inside the \verb|User| folder too, as it is compatible with all STM32 microcontrollers that support \gls{USB}.
GEX is designed to be modular and easy to extend. The user-facing functionality is composed of independent software modules, called \textit{functional blocks} or \textit{units}, which can be configured by the user to fit their application needs. Units implement low-level logic to work with hardware peripherals of the microcontroller, and expose this functionality to the client application, running on the \gls{PC}, through a communication interface. A diagram showing the entire stack, from the user application down to hardware peripherals, is shown in \cref{fig:conceptual}.
\begin{figure}[h]
\centering
\includegraphics[scale=1]{img/conceptual.pdf}
\caption[GEX conceptual overview]{\label{fig:conceptual}The ``GEX stack'', from a user application down to hardware}
\end{figure}
When we work with GEX, it is through units. The platform without units would be just an empty shell, the bare core framework; this underlying system will be described in \cref{sec:coreframework}. We will explore the individual units in \cref{sec:units-overview}, after going through the hardware realizations in \cref{sec:hwreal} and covering the communication protocol in \cref{sec:tinyframe}.
\section{Physical User Interface}
The firmware can be flashed to a STM32 development board, or a custom \gls{PCB}. The particulars of those form factors will be discussed in \cref{sec:hwreal}.
\begin{figure}[h]
\centering
\includegraphics[scale=.95]{img/users-view.pdf}
\caption{\label{fig:users-view-of-gex}Physical user interface of a GEX module}
\end{figure}
\noindent
All GEX hardware platforms have some common characteristics (\cref{fig:users-view-of-gex}):
\begin{itemize}
\item\textbf{Power \gls{LED}} -- a simple indication that the board is powered on
\item\textbf{Status \gls{LED}} -- periodic flashing every 3\,s indicates correct operation, continuous light a software error\footnote{The microcontroller will then automatically restart within a few seconds due to a watchdog timeout.}; other light patterns may be shown as feedback to user actions or received commands
\item\textbf{Reset button} -- resets the \gls{MCU}; this is particularly useful during firmware development as an alternative to re-connecting the \gls{USB} cable
\item\textbf{Lock button} -- enables or disables access to configuration files through the virtual mass storage device
\item\textbf{Boot button} -- when held during restart (that is, while the reset button is released), the \gls{DFU} mode~\cite{usbif-dfu} is activated and a new firmware image can be flashed over the \gls{USB} connection using \mono{dfu-util}~\cite{dfu-util} or other firmware update application
\item\textbf{\gls{GPIO} header} -- a pin header exposing the \gls{MCU}'s \gls{GPIO} pins to be connected to external circuitry
\item\textbf{Communication interface} -- a connection to the host \gls{PC}; multiple options may be available to choose from, a direct \gls{USB} connection being the primary and always available option
\end{itemize}
\section{GEX-PC Connection}
\Cref{fig:users-view-of-gex} shows three ways to connect the module to a \gls{PC}. Each communication interface has its advantages and drawbacks, and is suitable for different use-cases.
\begin{itemize}
\item\textbf{Direct \gls{USB} connection}
This is the primary and most straightforward connection method. We use the \gls{CDCACM} and \gls{MSC}\gls{USB} classes to have the module appear as a virtual serial port and a mass storage device, as described in \cref{sec:usb-classes}. This method is the fastest of the three and works out-of-the-box on Linux and MacOS. On MS Windows it may require the right software driver to be installed and assigned manually\footnote{The STM32 Virtual COM port driver~\cite{stm-vcom} has been tested to work with GEX on MS Windows version 7 and 8, though it must be manually assigned to the device in the Device Manager. MS Windows 10 and later should support \gls{CDCACM} natively.}.
\item\textbf{Hardware \gls{UART}}
The hardware UART used as a communication interface is mapped to pins PA2 and PA3 to be compatible with the built-in \gls{USB}/\gls{UART} converter on STM32 Nucleo development boards. This interface is functionally identical to the \gls{CDCACM} connection, but the physical \gls{UART} is necessarily slower and does not implement flow control.
\item\textbf{Wireless connection}
A wireless connection is implemented using a radio module on the GEX board. To use it we need a counterpart, the \textit{wireless gateway}, which connects to the \gls{PC} using \gls{USB} with \gls{CDCACM}.
\end{itemize}
All three interfaces share the same binary communication protocol (see below). The wireless connection is an exception, as the gateway itself needs to be managed by the host application, and it can connect to more than one GEX board, necessitating an addressing scheme. Its modified protocol will be explained in \cref{sec:wireless}.
The \gls{USB} connection is always enabled first on start-up. GEX waits its for enumeration by the host \gls{PC}. When not enumerated in a few seconds, it concludes that the interface is not active and tries other enabled options. The wireless module, connected through \gls{SPI}, can be detected by reading one of its registers that should have a known value. A \gls{UART} interface cannot be tested so reliably, thus it is always considered active\footnote{A detection of the UART connection would be possible by measuring the Rx pin voltage, which should idle at a high level (here 3.3\,V). This was not implemented in the initial firmware version.}.
\section{Controlling GEX}
GEX is a platform providing access to low-level hardware to high-level applications. However, this ``high level'' is relative. As was shown in \cref{fig:conceptual}, the ``GEX stack'' ends with a \textit{client library}, a software library used by the \textit{user application}.
The communication protocol (one level lower in the diagram) is robust and well defined, making it possible to implement alternative client libraries in other programming languages, or for yet-unsupported platforms. This protocol is explained in \cref{sec:tinyframe}. The client library implements the communication protocol and gives the user application access to GEX units via \textit{unit handles}.
Any logic above the client library is in the hands of the user, which means that, to use GEX, they have to program a user application. Software libraries in languages C and Python are provided, and will be explained in \cref{sec:clientsw}. The Python library is easy to use even for beginner programmers, though we have to acknowledge that some users might not be familiar with programming at all. Making GEX more accessible to those users, e.g., through a graphical desktop application, is an appealing idea that is certainly worth pursuing in later work.
\section{Device Configuration}
The core framework and each of the units have a number of adjustable options determining their behavior. Those settings are internally stored in a binary form, but to make their adjustment comfortable for the user, they are mapped to text configuration files in the INI format.
\subsection{INI File Format}
INI files are, in our implementation, simple text files containing three basic syntax elements: comments, sections, and key-value entries. Sections group the key-value pairs into logical blocks, e.g., the configuration of individual units.
\begin{itemize}
\item\textbf{Comments} start with the hash symbol (\mono{\#}) and end at the end of line
\item\textbf{Sections} are textual labels enclosed in square brackets (\mono{[UNITS]})
\item\textbf{Key-value entries} are composed of a label, the equals sign, and its value; values may be text strings, decimal or hexadecimal numbers, lists of numbers separated by commas, or any other format appropriate for the particular key
\end{itemize}
\noindent
An example of the INI syntax is shown below.
\begin{inicode}
# comment
[section]
a = 123
b = 0xFF
port = A
pins = 1,2,3
\end{inicode}
\subsection{Configuration Files Structure}
The configuration is split into two files: \mono{UNITS.INI} and \mono{SYSTEM.INI}. The system configuration file has a simple structure and does not need much explanation beyond the comments already included in it; an example of its content is captured in \cref{lst:systemini}. The other file, as its name suggests, serves to configure GEX units.
# Create units by adding their names next to a type (e.g. DO=A,B),
# remove the same way. Reload to update the unit sections below.
# Digital output
DO=led
# Digital input with triggers
DI=btn1,btn2
# Neopixel RGB LED strip
NPX=
# I2C master
I2C=i2c
#...
[I2C:i2c@1]
# Peripheral number (I2Cx)
#...
\end{inicode}
\caption{\label{lst:unitsini}Part of the \mono{UNITS.INI} configuration file}
\end{listing}
The units file, illustrated in \cref{lst:unitsini}, is more complex, and \textit{interactive}. The top part, a \mono{[UNITS]} section, lists all available unit types. A unit is created by writing its name (an arbitrary label composed of letters, numbers, and underscore) next to the desired type. Each unit is then configured in a separate section lower in the file; however, how does one know what keys are needed for which unit? This problem is solved by \textit{interactivity} of the file.
After adding a unit name next to its type, we save the file. The disk temporarily disappears from the device list as the file's content updates. When we reopen the file, a section for the new unit will be appended for us to configure as necessary. To delete a unit, it is sufficient to remove its name from the list at the top and let the file regenerate the same way; the unit's section will disappear.
It is not uncommon that the entered (or default) configuration is invalid and the unit cannot be enabled. The error is reported by inserting a comment into the INI file, at the top of the section of the failing unit. This error message disappears when the problem is corrected.
Once we are satisfied with the configuration, it may be stored to the module's permanent memory. This is done by pushing the Lock button again, which also deactivates the virtual storage device.
It may be interesting to know that the configuration files can also be read and modified through the communication interface. A simple configuration editor (\cref{fig:gexync}) was developed to demonstrate this feature. Besides applications like this, we can use the programmatic configuration access to change GEX settings automatically by the client application. The changes may be persisted by a command, but that is not required, which lets us use them temporarily without modifying the stored configuration.
With the communication protocol clearly defined in \cref{sec:tinyframe,sec:units-overview}, respective \cref{sec:wireless} for the wireless gateway, the implementation of a client software is relatively straightforward. Two client libraries have been developed, in languages C and Python.
With the communication protocol clearly defined in \cref{sec:tinyframe,sec:units-overview}, respective \cref{sec:wireless} for the wireless gateway, the implementation of a client software is relatively straightforward. Two client libraries have been developed, in languages C and Python.
@ -191,19 +191,19 @@ The frame 0x20 (List Units) requests a list of all available units in the GEX mo
Frame types 0x10 (Unit Request) and 0x11 (Unit Report) are dedicated to messages sent to and by unit instances. Each has a fixed header (\textit{inside the payload}) followed by unit-specific data.
Frame types 0x10 (Unit Request) and 0x11 (Unit Report) are dedicated to messages sent to and by unit instances. Each has a fixed header (\textit{inside the payload}) followed by unit-specific data.
Unit requests deliver a message from the host to a unit instance. Unit drivers implements different commands, each with its own payload structure. The frame 0x10 (Unit Request) has the following structure:
Unit requests deliver a message from the host to a unit instance. Unit drivers implements different commands, each with its own payload structure. The frame 0x10 (Unit Request) has the following structure:
\cfield{u8} command number, handled by the unit driver
\cfield{u8} command number, handled by the unit driver
\cfield{u8[]} command payload, handled by the unit driver; its size and content depend on the unit driver and the particular command number
\cfield{u8[]} command payload, handled by the unit driver; its size and content depend on the unit driver and the particular command number, as listed in \cref{sec:units-overview}
\end{boxedpayload}
\end{boxedpayload}
The most significant bit of the command byte (0x80) has a special meaning: when set, the message delivering routine responds with 0x00 (Success) after the command completes, unless an error occurred. That is used to get a confirmation that the message was delivered and the module operates correctly (as opposed to, e.g., a lock-up resulting in a watchdog reset). Requests which normally generate a response (e.g., reading a value from the unit) should not be sent with this bit set. As a result of this special treatment of the highest bit, there can be only 127 different commands per unit.
The most significant bit of the command byte (0x80) has a special meaning: when set, the message delivering routine responds with 0x00 (Success) after the command completes, unless an error occurred. That is used to get a confirmation that the message was delivered and the module operates correctly (as opposed to, e.g., a lock-up resulting in a watchdog reset). Requests which normally generate a response (e.g., reading a value from the unit) should not be sent with this flag, as that would produce two responses at once.
Several unit types can produce asynchronous events, such as reporting a pin change or a triggering condition. The event is timestamped and sent with a frame type 0x11 (Unit Report):
Several unit types can produce asynchronous events, such as reporting a pin change or a triggering condition. The event is timestamped and sent with a frame type 0x11 (Unit Report):
@ -211,7 +211,7 @@ Several unit types can produce asynchronous events, such as reporting a pin chan
\cfield{u8} unit callsign
\cfield{u8} unit callsign
\cfield{u8} report type, defined by the unit driver
\cfield{u8} report type, defined by the unit driver
\cfield{u64} event time (microseconds since power-on)
\cfield{u64} event time (microseconds since power-on)
\cfield{u8[]} report payload; its size and content depend on the unit driver and the particular report type
\cfield{u8[]} report payload; similar to requests, the payload structure depends on the unit driver and the particular report type
@ -76,7 +76,7 @@ The V$_\mathrm{BUS}$ line supplies power to \textit{bus-powered} devices. \texti
\noindent More details about the electrical and physical connection may be found in~\cite{usb-nutshell}, sections \textit{Connectors} through \textit{Power}.
\noindent More details about the electrical and physical connection may be found in~\cite{usb-nutshell}, sections \textit{Connectors} through \textit{Power}.
\section{USB Classes}
\section{USB Classes}\label{sec:usb-classes}
This section explains the classes used in the GEX firmware. A list of all standard classes with a more detailed explanation can be found in~\cite{usb-class-list}.
This section explains the classes used in the GEX firmware. A list of all standard classes with a more detailed explanation can be found in~\cite{usb-class-list}.
