@ -65,15 +65,17 @@ It is clear that GEX does not pose a real threat to professional tools in terms
\section{Limitations}
\section{Limitations}
Our solution was designed with the expectation that the users are familiar with programming, and the requirement to develop a control script or application would not pose a problem. However, that is not universally true, and some users who might benefit from the platform will not be able to use it without a more user-friendly interface.
While the objective of the work was achieved, we are aware of several limitations that must be acknowledged so that they can be addressed in future work.
The STM32F072 microcontroller proved sufficient for the verification of the design, and in many cases provides a sufficient performance. Future expansion of the project is, unfortunately, limited by its flash and \gls{RAM} capacity, which are already used at about 85\,\%, based on the size of the binary image and the amount of allocated memory. Further, the STM32F072 has a Cortex-M0 core without any hardware support for floating point arithmetics, making any calculations with those numbers slow and requiring additional library functions, further increasing the firmware image size. The modules' power consumption has not been measured or optimized, but the inclusion of a wireless interface predisposes them to be used in battery-powered applications; the microcontroller choice and the power supply design may need to be revised for efficient battery operation.
The chosen STM32F072 microcontroller proved sufficient for the verification of the design, and in many cases provides a sufficient performance. Future expansion of the project is, unfortunately, limited by its flash and \gls{RAM} capacity, which are already used at about 85\,\%, based on the size of the binary image and the amount of allocated memory. Further, the STM32F072 has a Cortex-M0 core without any hardware support for floating point arithmetics, making any calculations with those numbers slow and requiring additional library functions, further increasing the firmware image size. The modules' power consumption has not been measured or optimized, but the inclusion of a wireless interface predisposes them to be used in battery-powered applications; the microcontroller choice and the power supply design may need to be revised for efficient battery operation.
Another limitation concerns the support software; our client libraries have not been tested on MS Windows, with Linux being the main development platform. The C library uses POSIX \gls{API} and UNIX-specific \gls{API} to manage the serial port, which is not portable. The Python library should work on MS Windows, provided libUSB is installed correctly. Further, on MS Windows prior to version 10, the virtual COM port connection requires the ``STM32 Virtual COM port driver'' to be manually installed and assigned in the Device Manager.
Another limitation concerns the support software; our client libraries have not been tested on MS Windows, with Linux being the main development platform. The C library uses POSIX \gls{API} and UNIX-specific \gls{API} to manage the serial port, which is not portable. The Python library should work on MS Windows, provided libUSB is installed correctly. Further, on MS Windows prior to version 10, the virtual COM port connection requires the ``STM32 Virtual COM port driver'' to be manually installed and assigned in the Device Manager.
Lastly, our solution was designed with the expectation that the users are familiar with programming, and the requirement to develop a control script or application would not pose a problem. However, that is not universally true, and some users who might benefit from the platform will not be able to use it without a more user-friendly interface.
\section{Future Development}
\section{Future Development}
Future development of the project might focus on expanding the number of supported hardware platforms in order to overcome the limitations of the used microcontroller model. In particular the STM32L series of low-power devices should be investigated, as it may be a better choice when a battery-based operation is needed. The hardware implementation should further be revised to maximize power efficiency.
Future development of the project should focus on expanding the number of supported hardware platforms in order to overcome the limitations of the used microcontroller model. In particular, the STM32L series of low-power devices should be investigated, as it may be a better choice when a battery-based operation is needed. The hardware implementation shall further be revised to maximize its power efficiency.
The client libraries need to be tested on other operating systems, and the C library may be expanded to provide a higher level of comfort to the user. The client library could further be reimplemented in more programming languages, such as Java or \CS, and graphical applications could be developed to make GEX more approachable to users less familiar with programming.
The client libraries need to be tested on other operating systems, and the C library may be expanded to provide a higher level of comfort to the user. The client library could further be reimplemented in more programming languages, such as Java or \CS, and graphical applications could be developed to make GEX more approachable to users less familiar with programming.
This chapter presents several applications of GEX. Those examples are the result of testing during the firmware development, and do not cover the full range of features, but should give the reader some idea about the possibilities.
This chapter presents several applications of GEX. Those examples are the result of evaluation during the firmware development and do not cover the full range of supported features; nonetheless, they should give the reader a good idea about the possibilities.
\section{Frequency Response Measurement}
\section{Frequency Response Measurement}
The frequency response of a filter may be measured with a combination of the \gls{ADC} and \gls{DAC} units. The \gls{DAC} is configured to generate a sine wave as a stimulus for the filter, while the \gls{ADC} captures the output waveform. By varying the generated frequency and measuring the amplitude, we obtain a frequency response of the filter. If we measured both the input and output, we could calculate the phase shift and produce a real Bode diagram.
\caption{GEX Zero measuring a passive high-pass RC filter}
\label{fig:demofilter}
\end{wrapfigure}
\cref{fig:demofilter} depicts a simple test setup with a passive RC filter; its characteristics in the low-pass and high-pass configuration, as obtained with GEX, are shown in \cref{fig:demofilter_cap}.
The frequency response of a filter may be measured with a combination of the \gls{ADC} and \gls{DAC} units. The \gls{DAC} is configured to generate a sine wave as a stimulus for the filter, while the \gls{ADC} captures the output waveform. By varying the generated frequency and measuring the amplitude, we obtain a frequency response of the filter. If we measured both the input and output, we could calculate the phase shift and produce a real Bode diagram.
\begin{figure}[h]
\cref{fig:demofilter} depicts a simple test setup with a passive RC filter with a 100\,nF capacitor and a 470\,$\Omega$ resistor; its characteristics in the low-pass and high-pass configuration, as obtained by GEX, are shown in \cref{fig:demofilter_cap}. The irregularity at higher frequencies is likely caused by noise and nonlinearities in the \gls{DAC} output.
@ -23,7 +23,7 @@ The Raspberry Pi (\cref{fig:rpi}) is a low-cost minicomputer targeted at school
The board's \gls{GPIO} header, a row of pins supporting features such as \gls{SPI}, \gls{I2C}, \gls{UART}, or \gls{PWM}, directly accessible by user applications running on the minicomputer, was one of the inspirations behind GEX.
The board's \gls{GPIO} header, a row of pins supporting features such as \gls{SPI}, \gls{I2C}, \gls{UART}, or \gls{PWM}, directly accessible by user applications running on the minicomputer, was one of the inspirations behind GEX.
The Raspberry Pi's functionality clearly overlaps with features we wish to support in GEX. Its processor is powerful enough for a regular \gls{OS} with a graphical user interface, and after attaching a display and a keyboard, it can be used as a \gls{PC}. However, when we have a more powerful computer available and only want to extend it with the \gls{GPIO} header, having to use the Raspberry Pi seems inconvenient; this might be overcome with the use of screen sharing or \gls{SSH}, but a low-complexity solution like GEX certainly has its appeal.
The Raspberry Pi's functionality clearly overlaps with features we wish to support in GEX. Its processor is powerful enough for a regular \gls{OS} with a graphical user interface, and after attaching a display and a keyboard, it can be used as a \gls{PC}. However, when we have a more powerful computer available and only want to extend it with the \gls{GPIO} header, having to use the Raspberry Pi seems inconvenient; this might be overcome with the use of screen sharing or \gls{SSH}, but a low-cost and low-complexity solution like GEX certainly has its appeal.
\section{Bus Pirate}
\section{Bus Pirate}
@ -36,19 +36,15 @@ The Raspberry Pi's functionality clearly overlaps with features we wish to suppo
%http://dangerousprototypes.com/blog/about/
%http://dangerousprototypes.com/blog/about/
% Dangerous Prototypes and manufactured by Seeed Studio\todo{link}
% Dangerous Prototypes and manufactured by Seeed Studio\todo{link}
Bus Pirate~\cite{buspirate} is a USB-attached device providing access to hardware interfaces like \gls{SPI}, \gls{I2C}, \gls{USART}, and 1-Wire, as well as features like frequency measurement and direct pin access. The board aims to make it easy for users to familiarize themselves with unknown chips and modules; it also provides a range of programming interfaces to program microcontrollers and memory chips. The board communicates with the \gls{PC} using an FTDI USB-serial adapter.
Bus Pirate~\cite{buspirate} is a USB-attached device providing access to hardware interfaces like \gls{SPI}, \gls{I2C}, and \gls{USART}, as well as features like frequency measurement and direct pin access. The board aims to make it easy for users to familiarize themselves with unknown chips and modules; it also provides a range of programming interfaces to program microcontrollers and memory chips. In its scope, the Bus Pirate is similar to GEX. It can further be scripted and controlled from the PC, connects to \gls{USB}, and provides a wide selection of hardware interfaces.
Bus Pirate is open source and is, in its scope, similar to GEX. It can be scripted and controlled from the PC, connects to USB and provides a wide selection of hardware interfaces.
The board is based on a PIC16 microcontroller running at 32\,MHz, connected to the \gls{PC} using a FTDI \gls{USB}/\gls{UART} adapter. Its \gls{ADC} only has a resolution of 10 bits (1024 levels), and there is no \gls{DAC} available on the chip, which makes applications that require a varied output voltage more difficult to implement. Another limitation of the board is its low number of \gls{GPIO} pins; this, however, is not a hindrance to its main purpose as a bus analyzer and gateway.
The board is based on a PIC16 microcontroller running at 32\,MHz. Its \gls{ADC} only has a resolution of 10 bits (1024 levels). There is no \gls{DAC} available on the chip, which makes applications that require a varied output voltage more difficult to implement. Another limitation of the board is its low number of \gls{GPIO} pins, which may be insufficient for certain applications; this, however, is not a hindrance to its main purpose as a bus analyzer and tinkering tool.
The board can be purchased for a price similar to the Raspberry Pi, which is still affordable but more expensive that one might expect. This is likely given by the much lower volume of production.
The Bus Pirate is available for purchase at around 30\,USD (at the time of writing), a price comparable to some Raspberry Pi models.
\section{Professional DAQ Modules}
\section{Professional DAQ Modules}
Various professional tools that would fulfill our needs exist on the market, but their high price makes them inaccessible for users with a limited budget, such as hobbyists or students who would like to keep such a device for personal use. An example is the National Instruments \IIC/SPI Interface Device which also includes several \gls{GPIO} lines, their USB DAQ module, or some of the Total Phase \IIC/SPI gadgets (\cref{fig:profidaq}).
Various professional tools that would fulfill our needs exist on the market, but their high price makes them inaccessible for users with a limited budget, such as hobbyists or students who would like to keep such a device for personal use. An example is the National Instruments (NI) \IIC/SPI Interface Device, which also includes several \gls{GPIO} lines, the NI USB DAQ module, or some of the Total Phase \IIC/SPI gadgets (\cref{fig:profidaq}). GEX could further, in some cases, replace bench multimeters, counters, or waveform generators.
The performance GEX can provide may not always match that of those professional tools, but in many cases it will be a sufficient substitute at a fraction of the cost.
\chapter{The FAT16 File System and Its Emulation}\label{sec:fat16}
\chapter{The FAT16 File System and Its Emulation}\label{sec:fat16}
A \gls{FS} is used by GEX to provide the user comfortable access to the configuration files. By emulating a mass storage \gls{USB} device, the module appears as a thumb drive on the host \gls{PC}, and the user can edit its configuration using their preferred text editor. The FAT16 file system was selected for its simplicity and good cross-platform support~\cite{os-support-table}.
A \gls{FS} is used by GEX to provide the user with comfortable access to the configuration files. By emulating a mass storage \gls{USB} device, the module appears as a thumb drive on the host \gls{PC}, and the user can edit the files using their preferred text editor. The FAT16 file system was selected for its simplicity and good cross-platform support~\cite{os-support-table}.
Three variants of the \gls{FAT} file system exist: FAT12, FAT16, and FAT32. FAT12 was used on floppy disks and is similar to FAT16, except for additional size constraints and a \gls{FAT} entry packing scheme. FAT16 and FAT32 are FAT12's later developments from the time when hard disks became more common and the old addressing scheme could not support their larger capacity.
Three variants of the \gls{FAT} file system exist: FAT12, FAT16, and FAT32. FAT12 was used on floppy disks and is similar to FAT16, except for additional size constraints and a \gls{FAT} entry packing scheme. FAT16 and FAT32 are FAT12's later developments from the time when high-capacity hard disks became more common and the old addressing scheme proved insufficient.
This chapter will explain the structure of FAT16 and the challenges faced when trying to emulate it without a physical storage medium. A more detailed overview of the file system can be found in literature~\cite{ms-fat,fat16-brainy,fat16-maverick,fat16-phobos,fat-whitepaper} consulted during the GEX firmware development, with the Microsoft white paper~\cite{fat-whitepaper} giving the most complete description.
This chapter will explain the general structure of FAT16 and the challenges faced when trying to emulate it without a physical storage medium. A more detailed overview of the file system can be found in literature consulted during the GEX firmware development~\cite{ms-fat,fat16-brainy,fat16-maverick,fat16-phobos,fat-whitepaper}, with the Microsoft white paper~\cite{fat-whitepaper} giving the most detailed description.
\section{The General Structure of the FAT File System}
\section{The General Structure of the FAT File System}
The storage medium is organized into \textit{sectors} (or \textit{blocks}), usually 512 bytes long; that is the smallest addressing unit used by the file system. The disk starts with a \textit{boot sector}, also called the \gls{MBR}, followed by optional reserved sectors, one or two copies of the file allocation table, and the root directory. All disk areas are aligned to a sector boundary:
The storage medium is organized into \textit{sectors} (or \textit{blocks}), usually 512 bytes long; those are the smallest addressing units used by the file system. The disk starts with a \textit{boot sector}, also called the \gls{MBR}, followed by optional reserved sectors, one or two copies of the file allocation table, and the root directory. \Cref{tab:fat16_disk_areas} shows an overview of the dis areas.
\begin{table}[H]
\begin{table}[H]
\centering
\centering
@ -29,13 +29,13 @@ The storage medium is organized into \textit{sectors} (or \textit{blocks}), usua
\subsection{Boot Sector}
\subsection{Boot Sector}
This is a 1-sector structure which holds the \gls{OS} bootstrap code for bootable disks. The first 3 bytes are a jump instruction to the actual bootstrap code located later in the sector. What matters to us when implementing the file system is that the boot sector also contains data fields describing how the disk is organized, what file system is used, who formatted it, etc. The size of the \gls{FAT} and the root directory is defined here. The exact structure of the boot sector can be found in either of~\cite{ms-fat,fat16-brainy,fat16-maverick,fat16-phobos,fat-whitepaper} or in the attached GEX source code.
The ``boot sector'' is a 1-sector structure which holds the \gls{OS} bootstrap code for bootable disks. The first 3 bytes are a jump instruction to the actual bootstrap code located further in the sector. What matters to us when implementing the file system is that the boot sector also contains data fields describing how the disk is organized, what file system is used, who formatted it, etc. The size of the \gls{FAT} and the root directory is defined here. The exact structure of the boot sector can be found in literature or in the attached GEX source code.
\subsection{File Allocation Table}
\subsection{File Allocation Table}
The data area of the disk is organized in clusters, logical allocation units composed of groups of sectors. The use of a larger allocation unit allows the system to use shorter addresses and thus support a larger disk capacity.
The data area of the disk is divided into clusters, logical allocation units composed of groups of sectors. The use of a larger allocation unit allows the system to use shorter addresses and thus support a larger disk capacity.
The \gls{FAT} acts as a look-up table combined with linked lists. In FAT16, it is organized in 16-bit fields, each corresponding to one cluster. The first two entries in the allocation table are reserved and hold special values set by the disk formatter and the host \gls{OS}: a ``media descriptor'' 0xFFF8 and a ``clean/dirty flag'' 0xFFFF/0x3FFF.
The \gls{FAT}, residing before the data area, acts as a look-up table combined with linked lists. In FAT16, it is organized in 16-bit fields, each corresponding to one cluster. The first two entries in the allocation table are reserved and hold special values set by the disk formatter and the host \gls{OS}: a ``media descriptor'' 0xFFF8 and a ``clean/dirty flag'' 0xFFFF/0x3FFF.
Files can span multiple clusters; each \gls{FAT} entry either holds the address of the following file cluster, or a special value:
Files can span multiple clusters; each \gls{FAT} entry either holds the address of the following file cluster, or a special value:
@ -49,9 +49,9 @@ The bad cluster mark, 0xFFF7, is used for clusters which are known to corrupt da
\subsection{Root Directory}
\subsection{Root Directory}
A directory is a record on the disk that can span several clusters and holds information about the files and sub-directories contained in it. The root directory has the same structure as any other directory; the difference lies in the fact that it is allocated with a fixed position and size when the disk is formatted, whereas other directories are stored in the same way as ordinary files and their capacity can be increased by simply expanding to another cluster.
A directory is a record on the disk that can span several clusters and holds information about the files and sub-directories contained in it. The root directory has the same structure as any other directory; the difference lies in the fact that it is allocated with a fixed position and size when the disk is formatted, whereas other directories are stored in the same way as ordinary files, and their capacity can be increased by simply expanding to another cluster.
Directories are organized in 32-byte entries representing individual files. \Cref{tab:fat16_dir_entry} shows the structure of one such entry. The name and extension fields form the well-known ``8.3'' file name format known from MS DOS\footnote{``8.3'' refers to the byte size of the name and extension fields in the directory entry.}. Longer file names are encoded using the \gls{LFN} scheme~\cite{fat-lfn} as special hidden entries stored in the directory table alongside the regular ``8.3'' ones kept for backward compatibility.
Directories are organized into 32-byte entries representing individual files; \cref{tab:fat16_dir_entry} shows the structure of one such entry. The name and extension fields form the ``8.3'' file name format known from MS DOS\footnote{``8.3'' refers to the byte size of the fields in the directory entry}. Longer file names are encoded using the \gls{LFN} scheme~\cite{fat-lfn} as special hidden entries stored in the directory table alongside the regular ``8.3'' ones that are kept for backward compatibility.
\begin{table}
\begin{table}
\centering
\centering
@ -84,7 +84,7 @@ The first byte of the file name has special meaning:
The attributes field contains flags such as \textit{directory}, \textit{volume label}, \textit{read-only} and \textit{hidden}. Volume label is a special entry in the root directory defining the disk's label shown by the host \gls{OS}. A file with the directory bit set is actually a pointer to a subdirectory, meaning that when we open the linked cluster, we will find another directory table.
The attributes field contains flags such as \textit{directory}, \textit{volume label}, \textit{read-only} and \textit{hidden}. Volume label is a special entry in the root directory defining the disk's label shown by the host \gls{OS}. A file with the directory bit set is actually a pointer to a subdirectory, meaning that when we open the linked cluster, we will find another directory table.
\Cref{fig:fat_example} shows a possible organization of the GEX file system with two INI files, one spanning two clusters, the other being entirely inside one. The clusters need not be used completely; the exact sizes are stored in the files' directory entries.
\Cref{fig:fat_example} shows a possible organization of the GEX file system with two INI files, one spanning two clusters, the other being entirely inside one. The clusters need not be used completely, as the exact sizes are stored in the files' directory entries.
\begin{figure}[h]
\begin{figure}[h]
\centering
\centering
@ -95,7 +95,7 @@ The attributes field contains flags such as \textit{directory}, \textit{volume l
\section{FAT16 Emulation}
\section{FAT16 Emulation}
The FAT16 file system is relatively straightforward to implement. However, it is not practical or even possible to keep the entire file system in memory on a small microcontroller like our STM32F072. This means that we have to generate and parse disk sectors and clusters on-demand, when the host reads or writes them. The STM32 \gls{USB} Device library helpfully implements the \gls{MSC} and provides \gls{API} endpoints to which we connect our file system emulator. Specifically, those are requests to read and write a sector, and to the read disk's status and its parameters, such as the capacity.
The FAT16 file system is relatively straightforward to implement. However, it is not practical or even possible to keep the entire file system in memory on a small microcontroller like our STM32F072. This means that we have to generate and parse disk sectors and clusters on-demand, when the host reads or writes them. The STM32 \gls{USB} Device library helpfully implements the \gls{MSC} and provides \gls{API} endpoints to which we connect our file system emulator. Specifically, those are requests to read and write a sector, and to read the disk status and parameters, such as capacity.
\subsection{DAPLink Emulator}
\subsection{DAPLink Emulator}
@ -103,11 +103,11 @@ A FAT16 emulator was developed as part of the open-source \mbed DAPLink project~
The GEX firmware adapts several parts of the DAPLink code, optimizing its \gls{RAM} usage and porting it to work with FreeRTOS. The emulator source code is located in the \mono{User/vfs} folder of the GEX repository; the original Apache 2.0 open-source software license headers, as well as the file names, have been retained.
The GEX firmware adapts several parts of the DAPLink code, optimizing its \gls{RAM} usage and porting it to work with FreeRTOS. The emulator source code is located in the \mono{User/vfs} folder of the GEX repository; the original Apache 2.0 open-source software license headers, as well as the file names, have been retained.
As shown in \cref{tab:fat16_disk_areas}, the disk consists of several areas. The boot sector is immutable and can be loaded from the flash memory when requested. The handling of the other disk areas (\gls{FAT}, data area) depends on the type of access: read or write.
As shown in \cref{tab:fat16_disk_areas}, a FAT16-formatted disk consists of several areas. The boot sector is immutable and can be loaded from the flash memory when requested. The handling of the other disk areas (\gls{FAT}, data area) depends on the type of access: read or write.
\subsection{Read Access}
\subsection{Read Access}
The user can only read files that already exist on the disk; in our case, \verb|UNITS.INI| and \verb|SYSTEM.INI|. Those files are dynamically generated from the binary settings storage and, conversely, parsed as a byte stream without ever existing in their full form. This fact makes our task more challenging, as the file size cannot be easily measured and there is no obvious way to read a sector from the middle of a longer file. We solve this by implementing two additional functions in the INI file generation routine: a \textit{read window} and a \textit{dummy read mode}.
The user can only read files that already exist on the disk, in our case INI configuration files. Those files are dynamically generated from the binary settings storage and, conversely, parsed as a byte stream without ever existing in their full form. This fact makes our task more challenging, as the file size cannot be easily measured and there is no obvious way to read a sector from the middle of a longer file. We solve this by implementing two additional functions in the INI file generation routine: a \textit{read window} and a \textit{dummy read mode}.
A read window is a specification of the byte range we wish to generate. The INI generator discards bytes before the start of the read window, writes those inside the window to a holding buffer, and stops the end of the window is reached. This lets us extract a sector from anywhere in a file. The second function, dummy read, is tied to the window function: we set the start index so high that it is never reached (e.g., 0xFFFFFFFF), and have the generator count discarded characters. This character counter holds the full file size once the generation is completed.
A read window is a specification of the byte range we wish to generate. The INI generator discards bytes before the start of the read window, writes those inside the window to a holding buffer, and stops the end of the window is reached. This lets us extract a sector from anywhere in a file. The second function, dummy read, is tied to the window function: we set the start index so high that it is never reached (e.g., 0xFFFFFFFF), and have the generator count discarded characters. This character counter holds the full file size once the generation is completed.
@ -154,7 +154,7 @@ The uncertain order of the written areas poses a problem when the file name has
A change to file's content is performed in a similar way to the creation of a new file, except instead of creating a new entry in the directory table, an existing one is updated with the new file size. The name of the file may be unknown until the content is written, but we could detect the file name by comparing the start sector with those of all files known to the virtual file system.
A change to file's content is performed in a similar way to the creation of a new file, except instead of creating a new entry in the directory table, an existing one is updated with the new file size. The name of the file may be unknown until the content is written, but we could detect the file name by comparing the start sector with those of all files known to the virtual file system.
In the case of GEX, the detection of a file name is not important; we expect only INI files to be written, and the particular file may be detected by its first section marker, such as \verb|[UNITS]| or \verb|[SYSTEM]|. Should a non-INI file be written by accident, the INI parser will likely detect a syntax error and discard it.
In the case of GEX, the detection of a file name is not important; we expect only INI files to be written, and the particular file may be detected by its first section marker. Should a non-INI file be written by accident, the INI parser will likely detect a syntax error and discard it.
It should be noted that a file could be updated only partially, skipping the clusters which remain unchanged, and there is also no guarantee regarding the order in which the file's sectors are written. A non-linear or partial file update is hard to process for the emulator, but it can be reliably detected and discarded. Fortunately, this host behavior has not been conclusively observed in practice, but a file update rarely fails for unknown reasons; this could be a possible cause.
It should be noted that a file could be updated only partially, skipping the clusters which remain unchanged, and there is also no guarantee regarding the order in which the file's sectors are written. A non-linear or partial file update is hard to process for the emulator, but it can be reliably detected and discarded. Fortunately, this host behavior has not been conclusively observed in practice, but a file update rarely fails for unknown reasons; this could be a possible cause.
FreeRTOS is a free, open-source real-time operating system kernel targeted at embedded systems; it has been ported to many different microcontroller architectures~\cite{freertos-ports-list} and it is the de-facto industry standard. The system is compact and designed to be easy to understand; it is written in C, with the exception of some architecture-specific routines which use assembly. A complete overview of its \gls{API} is available in the FreeRTOS reference manual~\cite{freertos-rm} and its guide book~\cite{freertos-book}.
FreeRTOS is a free, open-source real-time operating system kernel targeted at embedded systems; it has been ported to many different microcontroller architectures~\cite{freertos-ports-list} and it is the de-facto industry standard. The kernel is compact and designed to be easy to understand; it is written in C, with the exception of architecture-specific routines which may use assembly. A complete overview of its \gls{API} is available in the FreeRTOS reference manual~\cite{freertos-rm} and its guide book~\cite{freertos-book}.
FreeRTOS provides a task scheduler, forming the central part of the system, and implements queues, semaphores, and mutexes for message passing and the synchronization of concurrent tasks. Those features are summarily called \textit{synchronization objects}, or simply \textit{objects}.
FreeRTOS provides a task scheduler, forming the central part of the system, and implements queues, semaphores, and mutexes for message passing and the synchronization of concurrent tasks. Those features are summarily called \textit{synchronization objects}, or simply \textit{objects}.
The system is used in GEX for its synchronization objects that allow us to safely pass messages between interrupts and working threads, without deadlocks or race conditions; the particular usage of FreeRTOS features will be explained in \cref{sec:rtos_in_gex}. The built-in stack overflow protection helps us optimize task memory allocation\footnote{The stack monitor reports how much stack space was really used, so we can expand or shrink it as needed to make the application work reliably, without wasting memory that would never be used.}, and the heap allocator provided by FreeRTOS enables thread-safe dynamic allocation with a shared heap.
The system is used in GEX for its synchronization objects that allow us to safely pass messages between interrupts and working threads, without deadlocks or race conditions; the particular usage of FreeRTOS features will be explained in \cref{sec:rtos_in_gex}. A built-in stack overflow protection (\cref{sec:stackprot}) helps us optimize task memory allocation\footnote{The stack monitor reports how much stack space was really used, so we can expand or shrink it as needed to make the application work reliably, without wasting memory that would never be used.}, and the heap allocator provided by FreeRTOS enables thread-safe dynamic allocation with a shared heap.
\section{Basic FreeRTOS Concepts and Functions}
\section{Basic FreeRTOS Concepts and Functions}
@ -12,15 +12,15 @@ The system is used in GEX for its synchronization objects that allow us to safel
Threads in FreeRTOS are called \textit{tasks}. Each task is assigned a memory area to use as its stack space, and a holding structure with its name, saved \textit{context}, and other metadata used by the kernel. A task context includes the program counter, stack pointer and other register values. Task switching is done by saving and restoring this context by manipulating the values on the stack before leaving an \gls{ISR}. The FreeRTOS website provides an example with the AVR port~\cite{freertos-task-switching} demonstrating how its internal functionality is implemented, including the context switch.
Threads in FreeRTOS are called \textit{tasks}. Each task is assigned a memory area to use as its stack space, and a holding structure with its name, saved \textit{context}, and other metadata used by the kernel. A task context includes the program counter, stack pointer and other register values. Task switching is done by saving and restoring this context by manipulating the values on the stack before leaving an \gls{ISR}. The FreeRTOS website provides an example with the AVR port~\cite{freertos-task-switching} demonstrating how its internal functionality is implemented, including the context switch.
At start-up the firmware initializes the kernel, registers tasks to run, and starts the scheduler. From this point onward the scheduler is in control and runs the tasks using a round robin scheme, always giving a task one tick of run time (usually 1\,ms) before interrupting it. Which task should run is determined primarily by their priority numbers, but there are other factors, as will be shown in \cref{sec:task_switching}.
At start-up, the firmware initializes the kernel, registers tasks to run, and starts the scheduler. From this point onward the scheduler is in control and runs the tasks using a Round Robin scheduling scheme, always giving a task one tick of run time (usually 1\,ms) before interrupting it. Which task should run is determined primarily by their priority numbers, but there are other factors, explained below.
\subsubsection{Task Run States}
\subsubsection{Task Run States}
Tasks can be in one of four states: Suspended, Ready, Blocked, and Running. The Suspended state does not normally occur in a task's life cycle, it is entered and left using API calls from the application. A task is in the Ready state when it can run, but is currently paused because a higher priority task is running. It enters the Running state when the scheduler switches to it. A Running task can wait for a synchronization object (e.g., a mutex) to be available; at this point it enters a Blocked state and the scheduler runs the next Ready task. When no tasks can run, the Idle Task takes control; it can either enter a sleep state to save power, or wait in a loop until another task is available. The Idle task is always either Ready or Running and has the lowest priority of all tasks.
Tasks can be in one of four states: Suspended, Ready, Blocked, and Running. The Suspended state does not normally occur in a task's life cycle, it is entered and left using \gls{API} calls from the application. A task is in the Ready state when it can run, but is currently paused because a higher priority task is running. It enters the Running state when the scheduler switches to it. A Running task can wait for a synchronization object (e.g., a mutex) to be available; at this point it enters a Blocked state and the scheduler runs the next Ready task. When no tasks can run, the Idle Task takes control; it can either enter a sleep state to save power, or wait in a loop until another task is available. The Idle task is always either Ready or Running and has the lowest priority of all tasks.
\subsubsection{Task Switching and Interrupts}\label{sec:task_switching}
\subsubsection{Task Switching and Interrupts}
Task switching occurs periodically in a timer interrupt, usually every 1\,ms; in \armcm chips this is typically the SysTick interrupt, a timer designed for this purpose that is included in the core itself and thus available on all derived platforms.
Task switching occurs periodically in a timer interrupt, usually every 1\,ms; in \armcm chips this is typically the SysTick interrupt, generated by a core timer that is specifically designed fro this purpose.
After one tick of run time, the Running task is paused and becomes Ready, or continues to run if no higher-priority task is available. If a higher-priority task waits for an object and this is made available in an \gls{ISR}, the running lower-priority task is paused and the waiting task resumes immediately. FreeRTOS defines interrupt-friendly variants of some of the \gls{API} functions intended for this purpose; however, only a subset of the \gls{API} is available in an \gls{ISR}, for example, it is not possible to use the delay function or wait for an object with a timeout, as the SysTick interrupt, incrementing the tick counter, has the lowest priority and could not run. This is by design, intended to prevent unexpected context switching in application interrupts.
After one tick of run time, the Running task is paused and becomes Ready, or continues to run if no higher-priority task is available. If a higher-priority task waits for an object and this is made available in an \gls{ISR}, the running lower-priority task is paused and the waiting task resumes immediately. FreeRTOS defines interrupt-friendly variants of some of the \gls{API} functions intended for this purpose; however, only a subset of the \gls{API} is available in an \gls{ISR}, for example, it is not possible to use the delay function or wait for an object with a timeout, as the SysTick interrupt, incrementing the tick counter, has the lowest priority and could not run. This is by design, intended to prevent unexpected context switching in application interrupts.
@ -38,12 +38,12 @@ FreeRTOS provides binary and counting semaphores, mutexes, and queues, which wil
\item\textbf{Mutexes} (locks) are similar to semaphores, but they must be taken and released in the same task. We use them to
\item\textbf{Mutexes} (locks) are similar to semaphores, but they must be taken and released in the same task. We use them to
guard an exclusive access to a resource, typically a hardware peripheral or a shared memory area. When a mutex is taken, any other tasks trying to take it too enter become Blocked. A Blocked task waiting for a mutex is resumed once this becomes available, at which point the task becomes its owner and is resumed.
guard an exclusive access to a resource, typically a hardware peripheral or a shared memory area. When a mutex is taken, any other tasks trying to take it too enter become Blocked. A Blocked task waiting for a mutex is resumed once this becomes available, at which point the task becomes its owner and is resumed.
\item\textbf{Queues} are used for passing messages between tasks, or from interrupts to tasks. Both sending and receiving of queue messages can block the task until the operation becomes possible. A queue handing task is often simply a loop which tries to read from the queue with an infinite timeout and processes the received data once the reading succeeds.
\item\textbf{Queues} are used for passing messages between tasks, or from interrupts to tasks. Both the sending and receiving of queue messages can block the task until the operation becomes possible. A queue handing task is often simply a loop which tries to read from the queue with an infinite timeout and processes the received data once the reading succeeds.
\end{itemize}
\end{itemize}
It must be noted that synchronization objects like mutexes and semaphores can help combat concurrent access only when used consistently and correctly. A locked mutex cannot guard against a rogue task accessing the protected resource without checking.
It must be noted that synchronization objects like mutexes and semaphores can help combat concurrent access only when used consistently and correctly. A locked mutex cannot guard against a rogue task accessing the protected resource without checking.
Each task in FreeRTOS is assigned a block of \gls{RAM} to use as its stack when it runs. This is where the stack pointer is restored to in the context switch. The stack pointer could move outside the designated area if the allocated space is insufficient; without countermeasures, this would mean that we are overwriting bytes in some unrelated memory structure, perhaps another task's stack memory.
Each task in FreeRTOS is assigned a block of \gls{RAM} to use as its stack when it runs. This is where the stack pointer is restored to in the context switch. The stack pointer could move outside the designated area if the allocated space is insufficient; without countermeasures, this would mean that we are overwriting bytes in some unrelated memory structure, perhaps another task's stack memory.
@ -25,7 +25,7 @@ The framework provides the following services to units:
\section{Unit Life Cycle and Internal Structure}\label{sec:units_function}
\section{Unit Life Cycle and Internal Structure}\label{sec:units_function}
GEX's user-facing functions, units, are implemented in \textit{unit drivers}. Those are independent modules in the firmware that the user can enable and configure, in one or more instances. In practice, we are limited by hardware constraints: i.e., there may be only one \gls{ADC} peripheral, or two \gls{SPI} ports. The assignment of those hardware resources to units is handled by the \textit{resource registry} (\cref{sec:res_allocation}).
GEX's user-facing functions, units, are implemented in \textit{unit drivers}. These are independent modules in the firmware that the user can enable and configure, in one or more instances. In practice, we are limited by hardware constraints: i.e., there may be only one \gls{ADC} peripheral, or two \gls{SPI} ports. The assignment of those hardware resources to units is handled by the \textit{resource registry} (\cref{sec:res_allocation}).
@ -108,11 +108,11 @@ It is evident that multiple units might need to use the same handler, even at th
The firmware is built around FreeRTOS (\cref{sec:freertos}) and a number of its synchronization objects and patterns are used to make its operation more robust.
The firmware is built around FreeRTOS (\cref{sec:freertos}) and uses a number of its synchronization objects and patterns to make the operation more robust.
\subsection{Message and Job Queue}
\subsection{Message and Job Queue}
The message and job queue, seen in \cref{fig:gex_internal}, is used to decouple asynchronous interrupts from message transmission. All three communication interfaces use interrupts for the asynchronous handling of incoming messages. The same interrupt handler receives an event after a transmission was completed. The queue ensures that messages can be received during the transmission of a large response that demands the use of multiple messages.
The message and job queue, shown in \cref{fig:gex_internal}, is used to decouple asynchronous interrupts from message transmission. All three communication interfaces use interrupts for the asynchronous handling of incoming messages. The same interrupt handler receives an event after a transmission was completed. The queue ensures that messages can be received during the transmission of a large response that demands the use of multiple messages.
GEX is designed to be modular and easy to extend. The user-facing functionality is composed of independent software modules, called \textit{functional blocks} or \textit{units}, which can be configured by the user to fit their application needs. Units implement low-level logic to work with hardware peripherals of the microcontroller, and expose this functionality to the client application, running on the \gls{PC}, through a communication interface. A diagram showing the entire stack, from the user application down to hardware peripherals, is shown in \cref{fig:conceptual}.
GEX is designed to be modular and easy to extend. The user-facing functionality is composed of independent software modules called \textit{functional blocks} or \textit{units}, which can be configured by the user to fit their application needs. Units implement low-level logic to work with hardware peripherals of the microcontroller, and expose this functionality to the client application, running on the \gls{PC}, through a communication interface. A diagram showing the entire stack, from the user application down to hardware peripherals, is shown in \cref{fig:conceptual}.
\begin{figure}[h]
\begin{figure}[h]
\centering
\centering
@ -8,11 +8,11 @@ GEX is designed to be modular and easy to extend. The user-facing functionality
\caption[GEX conceptual overview]{\label{fig:conceptual}The ``GEX stack'', from a user application down to hardware}
\caption[GEX conceptual overview]{\label{fig:conceptual}The ``GEX stack'', from a user application down to hardware}
\end{figure}
\end{figure}
When we work with GEX, it is through units. The platform without units would be just an empty shell, the bare core framework; this underlying system will be described in \cref{sec:coreframework}. We will explore the individual units in \cref{sec:units_overview}, after going through the hardware realizations in \cref{sec:hwreal} and covering the communication protocol in \cref{sec:tinyframe}.
When we work with GEX, it is through units. The platform without units would be just an empty shell, the bare core framework; this underlying system will be described in \cref{sec:coreframework}. We will explore the individual units in \cref{sec:units_overview}, after covering the communication protocol in \cref{sec:tinyframe}.
\section{Physical User Interface}
\section{Physical User Interface}
The firmware can be flashed to a STM32 development board, or a custom \gls{PCB}. The particulars of those form factors will be discussed in \cref{sec:hwreal}.
The firmware can be flashed to a STM32 development board, or a custom \gls{PCB}. The particulars of these form factors will be discussed in \cref{sec:hwreal}.
\begin{figure}[h]
\begin{figure}[h]
\centering
\centering
@ -21,7 +21,7 @@ The firmware can be flashed to a STM32 development board, or a custom \gls{PCB}.
\end{figure}
\end{figure}
\noindent
\noindent
All GEX hardware platforms have some common characteristics (\cref{fig:users_view_of_gex}):
All GEX hardware platforms have some common characteristics, illustrated in \cref{fig:users_view_of_gex}:
\begin{itemize}
\begin{itemize}
\item\textbf{Power \gls{LED}} -- a simple indication that the board is powered on
\item\textbf{Power \gls{LED}} -- a simple indication that the board is powered on
@ -58,7 +58,7 @@ The \gls{USB} connection is always enabled first on start-up. GEX waits its for
GEX is a platform providing access to low-level hardware to high-level applications. However, this ``high level'' is relative. As was shown in \cref{fig:conceptual}, the ``GEX stack'' ends with a \textit{client library}, a software library used by the \textit{user application}.
GEX is a platform providing access to low-level hardware to high-level applications. However, this ``high level'' is relative. As was shown in \cref{fig:conceptual}, the ``GEX stack'' ends with a \textit{client library}, a software library used by the \textit{user application}.
The communication protocol (one level lower in the diagram), which will be explained in \cref{sec:tinyframe}, may be implemented as part of the client library in any programming language and on any platform; in theory, it is even possible to control GEX from another microcontroller. The client library implements, besides the protocol itself, other higher-level logic, and gives the user access to individual units using an abstraction called \textit{unit handles}.
The communication protocol (one level lower in the diagram), which will be explained in \cref{sec:tinyframe}, may be implemented as part of the client library in any programming language and on any platform; it is even possible to control GEX from another microcontroller. The client library implements, besides the protocol itself, other high-level logic, and gives the user access to individual units using an abstraction called \textit{unit handles}.
Any logic above the client library is in the hands of the user, which means that, to use GEX, they have to program a user application. Proof-of-concept client libraries in languages C and Python are provided for this purpose, and will be explained in \cref{sec:clientsw}.
Any logic above the client library is in the hands of the user, which means that, to use GEX, they have to program a user application. Proof-of-concept client libraries in languages C and Python are provided for this purpose, and will be explained in \cref{sec:clientsw}.
@ -68,7 +68,7 @@ The core framework and each of the units have a number of adjustable options det
\subsection{INI File Format}
\subsection{INI File Format}
INI files are, in our implementation, simple text files containing three basic syntax elements: comments, sections, and key-value entries. Sections group the key-value pairs into logical blocks, e.g., the configuration of individual units.
INI files in our implementation are simple text files containing three basic syntactic elements: comments, sections, and key-value entries. Sections group the key-value pairs into logical blocks, and act as a prefix or a namespace for the keys.
\begin{itemize}
\begin{itemize}
\item\textbf{Comments} start with the hash symbol (\mono{\#}) and end at the end of line
\item\textbf{Comments} start with the hash symbol (\mono{\#}) and end at the end of line
@ -90,7 +90,7 @@ pins = 1,2,3
\subsection{Configuration Files Structure}
\subsection{Configuration Files Structure}
The configuration is split into two files: \mono{UNITS.INI} and \mono{SYSTEM.INI}. The system configuration file has a simple structure and does not need much explanation beyond the comments already included in it; an example of its content is captured in \cref{lst:systemini}. The other file, as its name suggests, serves to configure GEX units.
The configuration is split into two files: \mono{UNITS.INI} and \mono{SYSTEM.INI}. The system configuration file has a simple structure and does not need much explanation beyond the comments already included in it; an example of its content is captured in \cref{lst:systemini}. The other file, as its name suggests, configures GEX units.
\begin{listing}
\begin{listing}
\begin{inicode}
\begin{inicode}
@ -158,9 +158,9 @@ After adding a unit name next to its type, we save the file. The disk temporaril
It is not uncommon that the entered (or default) configuration is invalid and the unit cannot be enabled. The error is reported by inserting a comment into the INI file, at the top of the section of the failing unit. This error message disappears when the problem is corrected.
It is not uncommon that the entered (or default) configuration is invalid and the unit cannot be enabled. The error is reported by inserting a comment into the INI file, at the top of the section of the failing unit. This error message disappears when the problem is corrected.
Once we are satisfied with the configuration, it may be stored to the module's permanent memory. This is done by pushing the Lock button again, which also deactivates the virtual storage device.
Once we are satisfied with the configuration, it may be stored in the module's permanent memory. This is done by pushing the Lock button again, which also deactivates the virtual storage access.
It may be interesting to know that the configuration files can also be read and modified through the communication interface. A simple configuration editor (\cref{fig:gexync}) was developed to demonstrate this feature. Besides applications like this, we can use the programmatic configuration access to change GEX settings automatically by the client application. The changes may be persisted by a command, but that is not required, which lets us use them temporarily without modifying the stored configuration.
It may be interesting to know that the configuration files can also be read and modified through the communication interface. A simple configuration editor (\cref{fig:gexync}) was developed to demonstrate this feature. Besides editor applications like this, we can use the programmatic access to change GEX settings automatically by the user application. They may be persisted by a command, but that is not required, which lets us use them temporarily without modifying the stored configuration.
It has been proposed earlier in the text that STM32 Nucleo and Discovery development boards might serve as the hardware platform for this project. Indeed, a Discovery board with the STM32F072 was used to develop a major part of the GEX firmware, and the firmware remains compatible with it. This inexpensive board may be used to try GEX without the custom hardware.
It has been proposed earlier in the text that STM32 Nucleo and Discovery development boards might serve as the hardware platform for this project. Indeed, a Discovery board with the STM32F072 was used to develop a major part of the GEX firmware, and the firmware remains compatible with it. This inexpensive board may be used to try GEX without any custom hardware.
\subsection{Discovery STM32F072 Configuration and Pin Mapping}
\subsection{Discovery STM32F072 Configuration and Pin Mapping}
The Discovery board is fitted with four \glspl{LED} on \gls{GPIO} pins PC6 through PC9, in a compass arrangement. The ``north'' \gls{LED}, PC6, is used as the GEX status indicator. The ``User'' button, connected to PA0, is mapped as the GEX Lock button, controlling the settings storage.
The Discovery board is fitted with four \glspl{LED} on \gls{GPIO} pins PC6 through PC9, in a compass arrangement. The ``north'' \gls{LED}, PC6, is used as the status indicator. The ``User'' button, connected to PA0, is mapped as the Lock button, controlling the settings storage.
We advise the reader, as a potential user of the board, to review its schematic diagram (found in the documentation~\cite{disco-f072}) and ensure the solder-jumpers on the back side are configured correctly:
We advise the reader, as a potential user of the board, to review its schematic diagram (found in the documentation~\cite{disco-f072}) and ensure the solder-jumpers on the back side are configured correctly:
Hardware buses implemented in GEX are presented in this chapter. The description of each bus is accompanied by several examples of devices that can be interfaced with it. The reader is advised to consult the official specifications and particular devices' datasheets for additional details.
In this chapter we present the hardware buses to be supported by GEX. The description of each bus is accompanied by several examples of devices that can be interfaced with it. The reader is advised to consult the official specifications and the datasheets of individual sensors and other devices for additional details.
\section{UART and USART}\label{sec:theory_usart}
\section{UART and USART}\label{sec:theory_usart}
The \acrfull{USART} has a long history and is still in widespread use today. It is the protocol used in RS-232, which was once a common way of connecting modems, printers, mice and other devices to personal computers. RS-232 can be considered the ancestor of \gls{USB} in its widespread availability and use. \gls{UART} framing is also used in the industrial bus RS-485 and the automotive interconnect bus \gls{LIN}.
The \acrfull{USART} has a long history and is still widely used today. This is the frame format used in RS-232, which was once a common way of connecting modems, printers, mice and other devices to personal computers. RS-232 can be considered the ancestor of \gls{USB} in its popularity. \gls{UART} framing is also used in the industrial bus RS-485 and the automotive interconnect bus \gls{LIN}.
\begin{figure}[h]
\begin{figure}[h]
\centering
\centering
@ -16,7 +16,7 @@ The \acrfull{USART} has a long history and is still in widespread use today. It
\gls{USART}, as implemented by microcontrollers such as the STM32 family, is a two-wire full duplex interface that uses 3.3\,V or 5\,V logic levels. The data lines are in the high logical level when idle. A \gls{USART} frame, shown in \cref{fig:uart_frame}, starts by a start-bit (low level for the period of one bit) followed by \textit{n} data bits (typically eight), an optional parity bit, and a period of high level called a stop bit (or stop bits), dividing consecutive frames.
\gls{USART}, as implemented by microcontrollers such as the STM32 family, is a two-wire full duplex interface that uses 3.3\,V or 5\,V logic levels. The data lines are in the high logical level when idle. A \gls{USART} frame, shown in \cref{fig:uart_frame}, starts by a start-bit (low level for the period of one bit) followed by \textit{n} data bits (typically eight), an optional parity bit, and a period of high level called a stop bit (or stop bits), dividing consecutive frames.
RS-232 uses the \gls{UART} framing, but its levels are different: logical 1 is represented by negative voltages $-3$ to $-25$\,V and logical 0 uses the same range, but positive. To convert between RS-232 levels and \gls{TTL} (5\,V) levels, a level-shifting circuit such as the MAX232 can be used. In RS-232, the two data lines (Rx and Tx) are accompanied by \gls{RTS}, \gls{CTS}, and \gls{DTR}, which facilitate handshaking and hardware flow control. In practice, those additional signals are often unused or their function differs from their historical meaning; for instance, Arduino boards (using a USB-serial converter) use the \gls{DTR} line as a reset signal to automatically enter their bootloader for firmware flashing~\cite{arduinodtr}.
RS-232 uses the \gls{UART} framing, but its levels are different: logical 1 is represented by negative voltages $-3$ to $-25$\,V and logical 0 uses the same range, but positive. To convert between RS-232 levels and \gls{TTL} (5\,V) or 3.3\,V levels, a level-shifting circuit such as the MAX232 can be used. In RS-232, the two data lines (Rx and Tx) are accompanied by \gls{RTS}, \gls{CTS}, and \gls{DTR}, which facilitate handshaking and hardware flow control. In practice, those additional signals are often unused or their function differs from their historical meaning; for instance, Arduino boards (using a USB-serial converter) use the \gls{DTR} line as a reset signal to automatically enter their bootloader for firmware flashing~\cite{arduinodtr}.
In addition to communication buses, described in \cref{ch:hw_buses}, GEX implements several measurement and output functions that take advantage of the microcontroller's peripheral blocks, such as timers/counters and \gls{DAC}. The more complicated ones are described here; simpler functions, such as the raw \gls{GPIO} access, will be described later together with their control \gls{API}.
In addition to communication buses, described in \cref{ch:hw_buses}, GEX implements several measurement and output functions that take advantage of the microcontroller's peripheral blocks, such as timers/counters and \gls{DAC}. The more complicated ones are described here; simpler functions, such as the raw \gls{GPIO} access, will be described later when we look at the corresponding GEX's functional blocks.
@ -52,7 +52,7 @@ The system clock's frequency, which we use to measure pulse lengths and to gate
\section{Analog Signal Acquisition}\label{sec:theory_adc}
\section{Analog Signal Acquisition}\label{sec:theory_adc}
A very common need in experiments involving the measurement of physical properties is the acquisition of analog signals, respective voltages. These can be roughly divided into \gls{DC} and \gls{AC} or time-changing signals. Analog signals are converted to digital values using \glspl{ADC}. Several principles of analog signal measurement exist with different cost, speed, resolution, and many other factors which determine their suitability for a particular application.
A very common need in experiments involving the measurement of physical properties is the acquisition of analog signals (measured as voltage levels). These can be roughly divided into \gls{DC} and \gls{AC} or time-changing signals. Analog signals are converted to digital values using \glspl{ADC}. Several principles of analog signal measurement exist with different cost, speed, resolution, and many other factors which determine their suitability for a particular application.
\gls{DC} signals can be measured by taking several samples and calculating their average value; in the presence of mains interference (50\,Hz or 60\,Hz), it is advisable to spread those samples over the 20\,ms (resp. 16.7\,ms) time of one period, so that the interfering waveform cancels out. Time-changing signals can be captured by taking isochronous samples at a frequency conforming to the Nyquist theorem, that is, at least twice that of the measured signal. In practice, a frequency several times higher is preferred for a more accurate capture.
\gls{DC} signals can be measured by taking several samples and calculating their average value; in the presence of mains interference (50\,Hz or 60\,Hz), it is advisable to spread those samples over the 20\,ms (resp. 16.7\,ms) time of one period, so that the interfering waveform cancels out. Time-changing signals can be captured by taking isochronous samples at a frequency conforming to the Nyquist theorem, that is, at least twice that of the measured signal. In practice, a frequency several times higher is preferred for a more accurate capture.
With the communication protocol clearly defined in \cref{sec:tinyframe,sec:units_overview}, and \cref{sec:wireless} for the wireless gateway, the implementation of client libraries is relatively straightforward. Two have been developed as a proof-of-concept, in languages C and Python.
With the communication protocol clearly defined in \cref{sec:tinyframe,sec:units_overview}, and \cref{sec:wireless} for the wireless gateway, the implementation of client libraries is relatively straightforward. Two versions of the library have been developed as a proof-of-concept in languages C and Python.
\section{General Library Structure}
\section{General Library Structure}
@ -26,7 +26,7 @@ The structure of a GEX client library is in all cases similar:
\section{Python Library}
\section{Python Library}
The Python GEX library implements both a serial port access and raw access to \gls{USB} endpoints. Its development has been prioritized over the C library because of its potential to integrate with MATLAB, and because it promises to be the most convenient method to interact with GEX thanks to the ease-of-use that comes with the Python syntax. This library provides a high level \gls{API} above the individual unit types, removing the burden of building and parsing of the binary command payloads from the user.
The Python GEX library implements both serial port access and raw access to \gls{USB} endpoints. Its development has been prioritized over the C library because of its potential to integrate with MATLAB, and because it promises to be the most convenient method to interact with GEX thanks to the ease-of-use that comes with the Python syntax. This library provides a high level \gls{API} above the individual unit types, removing the burden of building and parsing of the binary command payloads from the user.
The library is composed of a \textit{transport} class, the core class \mono{gex.Client}, and unit classes (e.g., \mono{gex.I2C} or \mono{gex.SPI}).
The library is composed of a \textit{transport} class, the core class \mono{gex.Client}, and unit classes (e.g., \mono{gex.I2C} or \mono{gex.SPI}).
@ -92,7 +92,7 @@ First, a client instance is created, receiving the transport as an argument. We
\section{MATLAB integration}
\section{MATLAB integration}
The Python library can be accessed from MATLAB scripts thanks to MATLAB's two-way Python integration~\cite{matlabpy}. Controlling GEX from MATLAB may be useful when additional processing is required, e.g., with data from the \gls{ADC}; however, in many cases, an opensource alternative native to Python exists that could be used for the same purpose, such as the NumPy and SciPy libraries~\cite{numpyscipy}.
The Python library can be accessed from MATLAB scripts thanks to MATLAB's two-way Python integration~\cite{matlabpy}. Controlling GEX from MATLAB may be useful when additional processing is required, e.g., with data from the \gls{ADC}; however, in many cases, an open-source alternative native to Python exists that could be used for the same purpose, such as the NumPy and SciPy libraries~\cite{numpyscipy}.
The example in \cref{lst:matlab_api} (and \cref{fig:matlabpic}) demonstrates the use of MATLAB to calculate the frequency spectrum of an analog signal captured with GEX. The syntax needed to use the serial port transport (instead of a raw access to USB endpoints) is shown in a comment.
The example in \cref{lst:matlab_api} (and \cref{fig:matlabpic}) demonstrates the use of MATLAB to calculate the frequency spectrum of an analog signal captured with GEX. The syntax needed to use the serial port transport (instead of a raw access to USB endpoints) is shown in a comment.
@ -134,7 +134,9 @@ The example in \cref{lst:matlab_api} (and \cref{fig:matlabpic}) demonstrates the
The C library is more simplistic than the Python one; it supports only the serial port transport (\gls{UART} or \gls{CDCACM}) and does not implement asynchronous polling or the unit support drivers. The implemented features---the transport, a basic protocol handler, and payload building and parsing utilities---are sufficient for most applications, though less convenient than the Python library.
The C library is more simplistic than the Python one; it supports only the serial port transport (\gls{UART} or \gls{CDCACM}) and does not implement asynchronous polling or the unit support drivers. The implemented features---the transport, a basic protocol handler, and payload building and parsing utilities---are sufficient for most applications, though less convenient than the Python library.
This low-level library is intended for applications where the performance of the Python implementation is insufficient, or where an integration with existing C code is required. The full \gls{API} can be found in the library header files. A C version of the example Python script shown above, controlling an \gls{LED} matrix driver, is presented in \cref{lst:c_api_full}. The payloads in this example are represented as binary strings for simplicity. Two better methods of payload construction are available: using C structs, or taking advantage of the Payload Builder utility (bundled with TinyFrame).
This low-level library is intended for applications where the performance of the Python implementation is insufficient, or where an integration with existing C code is required. The full \gls{API} can be found in the library header files. A C version of the example Python script shown above, controlling an \gls{LED} matrix driver, is presented in \cref{lst:c_api_full}.
The payloads in this example are specified as binary strings, for simplicity. Two better methods of payload construction are available: using C structs, or taking advantage of the Payload Builder utility (bundled with TinyFrame).
When adding a new digital sensor or a module to a hardware project, we want to test it first, learn how to properly communicate with it, and confirm its performance. Based on this evaluation we decide whether the module matches our expectations, and learn how to properly connect it, which is needed for a successful \gls{PCB} design. In experimental setups, this connection may be the only thing we need. Sensor data can be collected immediately after gaining access to its communication interface, and the same applies to controlling actuators or other devices.
When adding a new digital sensor or a module to a hardware project, we want to test it first, learn how to properly communicate with it, and confirm its performance. Based on this evaluation we decide whether the module matches our expectations, and learn how to properly connect it. In experimental setups, this connection may be the only thing we need. Sensor data can be collected immediately after gaining access to its communication interface, and the same applies to controlling actuators or other devices.
A couple of well known hardware buses have established themselves as the standard ways to interface digital sensors and modules: \gls{SPI}, \gls{I2C} and \gls{USART} (\gls{UART} in asynchronous mode) are some of the most common ones, often accompanied by a few extra\gls{GPIO} lines for features such as Reset, Chip Enable, or Interrupt. There are exceptions where silicon vendors have developed proprietary communication protocols that continue to be used either for historical reasons, or because of their specific advantages. An example is the Dallas Semiconductor 1-Wire bus used in digital thermometers.
Several well-known hardware buses have established themselves as the standard ways to interface digital sensors and modules: \gls{SPI}, \gls{I2C} and \gls{USART} (\gls{UART} in asynchronous mode) are some of the most common ones, often accompanied by a few additional\gls{GPIO} lines for features such as Reset, Chip Enable, or Interrupt. There are exceptions where silicon vendors have developed proprietary communication protocols that continue to be used either for historical reasons, or because of their specific advantages. An example is the Dallas Semiconductor 1-Wire bus used in digital thermometers.
Moving to industrial and automotive environments, we encounter various fieldbuses, Ethernet, \gls{CAN}, current loop, \gls{HART}, \gls{LIN}, \gls{DALI}, RS-485 (e.g., for Modbus), \gls{mbus}, PLC-BUS, and others. Those typically use transceiver \glspl{IC} and other circuitry, such as \glspl{TVS}, signal filters, or galvanic isolation. They could be supported using add-on boards and additional firmware modules handling the protocol. For simplicity and to meet time constraints, the development of those boards and modules will be left for future expansions of the project.
Moving to industrial and automotive environments, we encounter various fieldbuses, Ethernet, \gls{CAN}, \gls{HART}, \gls{LIN}, \gls{DALI}, RS-485 (e.g., for Modbus), \gls{mbus}, PLC-BUS, and others. Those typically use transceiver \glspl{IC} and other circuitry, such as \glspl{TVS}, signal filters, or galvanic isolation.
Since trying to support everything would only distract us from developing a robust core system, we will focus only on the most common interfaces in this work, leaving the more specialized ones to future expansions and add-on boards.
\subsection{Analog Signal Acquisition}
\subsection{Analog Signal Acquisition}
Sometimes it is necessary to use a traditional analog sensor, capture a transient waveform, or to just measure voltage. GEX is meant to focus on digital interfaces, however giving it this capability makes it much more versatile. Nearly all microcontrollers include an \gls{ADC} which we can use to measure input voltages and, paired with a timer, to records signals varying in time.
Sometimes, it is necessary to use a traditional analog sensor, to capture a transient waveform, or to just measure voltage. While our main focus lies on digital interfaces, this capability will make the project much more versatile. Nearly all microcontrollers include an \gls{ADC} which we can use to measure input voltages, and, paired with a timer, to records signals varying in time.
Certain tasks, such as capturing transient effects on a thermocouple when inserted into a flame (an example from developing fire-proof materials) demand level triggering similar to that of oscilloscopes. The converter continuously measures the input voltage and a timed capture starts only after a set threshold is exceeded. This can be accompanied by a pre-trigger feature where the timed capture is continuously running and the last sample is always compared with the threshold, recording a portion of the historic records together with the following samples.
Certain tasks, such as capturing transient effects on a thermocouple when inserted into a flame (an example from developing fire-proof materials) demand level triggering, similar to that of oscilloscopes. The converter continuously measures the input voltage and a starts recording the samples only after a set threshold is exceeded. This can be accompanied by a pre-trigger feature where the immediate history is captured when the triggering condition occurs.
\subsection{Analog Signal Output}
\subsection{Analog Signal Output}
An analog signal can not only be measured, but it is often necessary to also generate it. This could serve as an excitation signal for an experiment, for instance to measure the characteristic curves of a diode or a transistor. Conveniently, we can at the same time use GEX's analog input to record the output.
An analog signal can not only be measured, but it is often necessary to also generate it. Pairing the analog output and input features, we can perform more complex measurements, such as measuring the frequency response of filters, or the characteristic curves of semiconductor devices.
Generating an analog signal is possible using a \gls{PWM} or by a dedicated digital-analog converter included in many microcontrollers. Higher frequencies or resolution can be achieved with a dedicated external \gls{IC}.
Generating an analog signal is possible using \gls{PWM} or by a dedicated \gls{DAC} included in many microcontrollers. Higher frequencies or resolution can be achieved with a dedicated external \gls{IC}.
\subsection{Logic Level Input and Output}
\subsection{Pulse Generation and Measurement}
We have covered some more advanced features, but skipped the simplest feature: direct access to \gls{GPIO} pins. Considering the latencies of \gls{USB} and the \gls{PC}'s \gls{OS}, this cannot be used reliably for ``bit banging''; however, we can still accomplish a lot with just changing logic levels---e.g., to control character \glspl{LCD}, or emulate some interfaces that include a clock line, like \gls{SPI}. As mentioned in \cref{sec:uses_digital_ifaces}, many digital sensors and modules use plain \glspl{GPIO} in addition to the communication bus for out-of-band signaling or features like chip selection or reset.
Some sensors have variable frequency or \gls{PWM} output. To capture those signals and convert them to a more useful digital value, we can utilize the external input functions of a timer/counter peripheral. Those timers have many possible configurations and can also be used for pulse counting or waveform generation.
\subsection{Pulse Generation and Measurement}
\subsection{Logic Level Input and Output}
Some sensors have a variable frequency or a \gls{PWM} output. To capture those signals and convert them to a more useful digital value, we can use the external input functions of a timer/counter in the microcontroller. Those timers have many possible configurations and can also be used for pulse counting or waveform generation.
We have covered some more advanced features, but skipped the simplest feature: direct access to \gls{GPIO} pins. Considering the latencies of \gls{USB} and the \gls{PC}'s \gls{OS}, this cannot be used reliably for ``bit banging'' custom protocols, unless they are tolerant to jitter, or very slow; however, we can still accomplish a lot with just changing logic levels---e.g., to control character displays with a parallel interface, or to emulate some interfaces that include a clock line. As mentioned in \cref{sec:uses_digital_ifaces}, many digital sensors and modules use plain \glspl{GPIO}, in addition to the communication bus, for out-of-band signaling or features like chip select or reset.
\section{Connection to the Host Computer}
\section{Connection to the Host Computer}
\subsection{Communication Interface}
\subsection{Communication Interface}
\gls{USB} shall be the primary way of connecting the module to a host \gls{PC}. Thanks to \gls{USB}'s flexibility, it can present itself as any kind of device or even multiple devices at once.
\gls{USB} shall be the primary way of connecting the module to a host \gls{PC}. Thanks to \gls{USB}'s flexibility, the \gls{MCU} can present itself as any kind of device, or even multiple devices at once.
The most straightforward method of interfacing the board is by passing binary messages in a fashion similar to \gls{UART}. We'll need a duplex connection to enable command confirmations, query-type commands and asynchronous event reporting. This is possible either using a ``Virtual COM port'' driver, or through raw access to the corresponding \gls{USB} endpoints. Using raw access avoids potential problems with the \gls{OS}'s driver interfering or not recognizing the device correctly; on the other hand, having GEX appear as a serial port makes it easier to integrate into existing platforms that have good serial port support (such as National Instruments LabWindows CVI or MATLAB).
The most straightforward method of interfacing the board is by passing binary messages in a fashion similar to \gls{UART}. This is possible either using a ``Virtual COM port'' driver, or through raw access to the corresponding \gls{USB} endpoints. Using raw access avoids potential problems with the \gls{OS}'s driver interfering or not recognizing the device correctly; on the other hand, having GEX appear as a serial port makes it easier to integrate into existing platforms that have good serial port support (such as National Instruments LabWindows~CVI, Visual C++ applications, or VBA macros).
A connection using a hardware \gls{UART} is also planned, as a fallback for boards without an USB connector or for platforms with no \gls{USB} connectivity. A wireless connection to the host PC should also be possible and work transparently in a similar way to the \gls{USB} or\gls{UART} connection.
GEX may be used with development boards lacking a ``User'' \gls{USB} connector, such as STM32 Nucleo. In this case, either the board can be customized, or we use the built-in \gls{USB}/\gls{UART} converter; that requires the communication interface to be available also through hardware\gls{UART}. Another use-case is in battery-powered setups where a wired access is not possible or practical. For those cases the ideal solution is a wireless connection.
\subsection{Configuration Files}
\subsection{Configuration Files}
The module must be easily reconfigurable. Given the settings are almost always going to be tied to the connected external hardware, it would be practical to have an option to store them permanently in the microcontroller's non-volatile memory.
The module must be easily reconfigurable. Given the settings are almost always going to be tied to the connected external hardware, it would be practical to have an option to store them permanently in the microcontroller's non-volatile (flash) memory.
We can load those settings into GEX using the serial interface, which also makes it possible to reconfigure it remotely when the wireless connection is used. With USB, we can additionally make the board appear as a mass storage device and expose the configuration as text files. This approach, inspired by \mbed's mechanism for flashing firmware images to development kits, avoids the need to create a configuration \gls{GUI}, instead using the built-in applications of the \gls{PC}\gls{OS} to view and edit files. Besides the configuration files, we can expose additional information, such as a README file with instructions, or a pin-out reference, as separate files on the virtual disk.
We can load those settings into GEX using the serial interface, which also makes it possible to reconfigure it remotely through the wireless connection. With USB, the board may additionally appear as a mass storage device and expose the configuration as text files. This approach, inspired by \mbed's mechanism for flashing firmware images to development kits, avoids the need to create a configuration \gls{GUI}, instead using the built-in applications of the \gls{PC}\gls{OS} to view and edit files.
\section{An Overview of Planned Features}
\section{An Overview of Planned Features}
@ -56,52 +58,51 @@ Summarizing the preceding discussion, we obtain the following list of features t
\item\textbf{Hardware interfacing functions}
\item\textbf{Hardware interfacing functions}
\begin{itemize}
\begin{itemize}
\item I/O pin direct access (read, write), pin change interrupt
\item I/O pin direct access (read, write), pin change interrupt
\item Analog input: voltage measurement, sampled capture
\item Analog input: voltage measurement, isochronous sampling
\item Analog output: static level, waveform generation
\item Analog output: DC level, waveform generation
\item Digital interfaces: \gls{SPI}, \gls{I2C}, \gls{UART}/\gls{USART}, and 1-Wire
%\item NeoPixel (addressable \gls{LED} strips)
\end{itemize}
\end{itemize}
\pagebreak[0]
\pagebreak[0]
\item\textbf{Communication with the host computer}
\item\textbf{Communication with the host computer}
\begin{itemize}
\begin{itemize}
\item\gls{USB} connection as virtual serial port or direct endpoint access
\item\gls{USB} connection as virtual COM port, or with direct endpoint access
\item Connection using plain \gls{UART}
\item Connection using plain \gls{UART}
\item Wireless attachment
\item Wireless link
\end{itemize}
\end{itemize}
\item\textbf{Configuration}
\item\textbf{Configuration}
\begin{itemize}
\begin{itemize}
\item Fully reconfigurable, temporarily or permanently
\item Fully reconfigurable, temporarily or permanently
\item Settings stored in INI files
\item Settings stored in INI files
\item File access through the communication interface or using a virtual mass storage
\item File access through the communication interface or a virtual mass storage device
\end{itemize}
\end{itemize}
\end{itemize}
\end{itemize}
\section{Microcontroller Selection}
\section{Microcontroller Selection}
As discussed in \cref{sec:expected_outcome}, this project will be based on microcontrollers from the STM32 family. The STM32F072 model was selected for the initial hardware and firmware design due to its low cost, advanced peripherals, and the availability of development boards. The firmware can be ported to other \glspl{MCU} later (e.g., to STM32L072, STM32F103 or STM32F303).
As discussed in \cref{sec:expected_outcome}, the project should be based on microcontrollers from the STM32 family. We chose the STM32F072 for the initial hardware and firmware design due to its low cost, advanced peripherals, and the availability of development boards. The firmware can be ported to other \glspl{MCU} later (e.g., to STM32L072, STM32F103 or STM32F303).
The STM32F072 is an \armcm device with 128\,KiB of flash memory, 16\,KiB of \gls{RAM} and running at 48\,MHz. It is equipped with a \gls{USB} Full Speed peripheral block, a 12-bit \gls{ADC} and \gls{DAC}, a number of general-purpose timers/counters, SPI, I$^2$C, and USART peripherals, among others. It supports crystal-less \gls{USB}, using the USB SOF packet for synchronization of the internal 48\,MHz RC oscillator; naturally, a real crystal resonator will provide better timing accuracy.
The STM32F072 is an \armcm device with 128\,KiB of flash memory, 16\,KiB of \gls{RAM}, and running at 48\,MHz. It is equipped with a \gls{USB} Full Speed peripheral block, 12-bit \gls{ADC} and \gls{DAC}, a number of general-purpose timers/counters, and peripheral blocks like SPI, I$^2$C, or USART. It supports a crystal-less \gls{USB} mode, using the \gls{USB} SOF packet to synchronize an internal 48\,MHz RC oscillator; naturally, a real crystal resonator will provide better timing accuracy.
To effectively utilize the time available for this work, only the STM32F072 firmware will be developed while making sure the planned expansion is as straightforward as possible.
To effectively utilize the time available for this work, only the STM32F072 firmware will be developed, while making sure the planned expansion to other modules and by adding additional features is as straightforward as possible.
While the GEX firmware can be used with existing evaluation boards from ST Microelectronics (\cref{fig:discovery}), we wish to design and realize a few custom hardware prototypes that will be smaller and more convenient to use.
While the GEX firmware can be used with existing evaluation boards from ST Microelectronics (\cref{fig:discovery}), we wish to design and realize a few custom hardware prototypes that will be smaller and more convenient to use.
Three possible form factors are drawn in \cref{fig:ff_sketches}. The use of a common connector layout and pin assignments, here Arduino and Raspberry Pi, makes it possible to reuse add-on boards from those platforms. When we copy the physical form factor of another product, in this example the Raspberry Pi Zero, we can further take advantage of existing enclosures designed for it.
Three possible form factors are drawn in \cref{fig:ff_sketches}. The use of a common connector layout and pin assignments, here Arduino and Raspberry Pi, makes it possible to reuse add-on boards from these platforms. Copying the physical shape and connector layout of another device further allows us to reuse existing enclosures designed for it.
\caption{\label{fig:repo_structure} General structure of the source code repository}
\caption{\label{fig:repo_structure} General structure of the source code repository}
\end{wrapfigure}
\end{wrapfigure}
Understanding the GEX source code layout is important before attempting to implement any changes or to port it to a different microcontroller type. The directory layout is shown in \cref{fig:repo_structure}.
Understanding the GEX source code layout is important before attempting to implement any changes or to port it to a different microcontroller model. The directory layout is shown in \cref{fig:repo_structure}.
The GEX core framework resides in the User folder, and units are defined in User/units. Each unit driver must be registered in the file \verb|platform.c|. The header file \verb|plat_compat.h| defines platform-specific constants and macros, defining parameters such as pin assignments or the clock speed. The User folder is actually a Git submodule called ``gex-core'' and is kept as a separate project; platform-specific customizations are managed using compile flags passed from the Makefile.
The GEX core framework resides in the User folder, and units are defined in User/units. Each unit driver must be registered in the file \verb|platform.c|. The header file \verb|plat_compat.h| defines platform-specific constants and macros, defining parameters such as pin assignments or the clock speed. The User folder is actually a Git submodule called ``gex-core'' and is kept as a separate project; platform-specific customizations are managed using compile flags passed from the Makefile.
This chapter presents an overview of the \acrfull{USB} Full Speed interface, with focus on the features used in the GEX firmware. \gls{USB} is a versatile and powerful interface which replaces several older technologies; for this reason its specification is very complex and going into all details is hardly possible. We will cover the basic principles and terminology of \gls{USB} and focus on the parts relevant for the GEX project. More information about the bus can be found in the official specification~\cite{usbif-spec}, related documents published by the USB Implementers Forum, and other on-line resources~\cite{usb-nutshell,usb-made-simple}.
This chapter presents an overview of the \acrfull{USB} Full Speed interface. \gls{USB} is a versatile and powerful interface which replaces several older technologies; for this reason its specification is very complex and going into all details is hardly possible. We will cover the basic principles and terminology of \gls{USB} and focus on the parts relevant for the GEX project. More information about the bus can be found in its official specification~\cite{usbif-spec}, related documents published by the USB Implementers Forum (USB-IF, the body maintaining the standard), and in on-line resources~\cite{usb-nutshell,usb-made-simple}.
\section{Basic Principles and Terminology}
\section{Basic Principles and Terminology}
@ -10,11 +10,11 @@ This chapter presents an overview of the \acrfull{USB} Full Speed interface, wit
\caption[USB hierarchical structure]{\label{fig:usb_hierarchy}The hierarchical structure of the USB bus}
\caption[USB hierarchical structure]{\label{fig:usb_hierarchy}The hierarchical structure of the USB bus}
\end{figure}
\end{figure}
\gls{USB} is a hierarchical bus (\cref{fig:usb_hierarchy}) with a single master (\textit{host}) and multiple slave devices. A \gls{USB} device that provides functionality to the host is called a \textit{function}~\cite{usb-function}.
\gls{USB} is a hierarchical bus (\cref{fig:usb_hierarchy}) with a single master (\textit{host}) and multiple slave devices. A \gls{USB} device that provides functionality to the host is called a \textit{function}.%~\cite{usb-function}.
\subsection{Pipes and Endpoints}
\subsection{Pipes and Endpoints}
Communication between the host and a function is organized into virtual channels called \textit{pipes} connecting to the device's \textit{endpoints}, identified by endpoint numbers. Pipes and endpoints are a high level abstraction of the connection between the host and a device (\cref{fig:usb_logical}).
Communication between the host and a function is organized into virtual channels called \textit{pipes} connecting to the device's \textit{endpoints}, identified by endpoint numbers. Pipes and endpoints are a high-level abstraction of the connection between the host and a function (\cref{fig:usb_logical}).
\begin{figure}[h]
\begin{figure}[h]
\centering
\centering
@ -22,7 +22,7 @@ Communication between the host and a function is organized into virtual channels
\caption{\label{fig:usb_logical}The logical structure of USB}
\caption{\label{fig:usb_logical}The logical structure of USB}
\end{figure}
\end{figure}
Endpoints can be either unidirectional or bidirectional; the direction from the host to a function is called OUT, the other direction (function to host) is called IN. A bidirectional endpoint is technically composed of IN and OUT endpoints with the same number. All transactions (both IN and OUT) are initiated by the host; functions have to wait for their turn. Endpoint 0 is bidirectional, always enabled, and serves as a \textit{control endpoint}. The host uses the control endpoint to read information about the device and configure it as needed.
Endpoints can be either unidirectional or bidirectional; the direction from the host to a function is called OUT, the other direction (function to host) is called IN. A bidirectional endpoint is technically composed of IN and OUT endpoints with the same number\footnote{By convention, IN endpoints use numbers with the most significant bit set (e.g., 0x82 for IN endpoint 2)}. All transactions (both IN and OUT) are initiated by the host; functions have to wait for their turn. Endpoint 0 is bidirectional, always enabled, and serves as a \textit{control endpoint}. The host uses the control endpoint to read information about the device and configure it as needed.
\subsection{Transfer Types}
\subsection{Transfer Types}
@ -30,39 +30,37 @@ There are four types of data transfers defined in \gls{USB}: control, bulk, isoc
\begin{itemize}
\begin{itemize}
\item\textit{Control} -- initial configuration after device plug-in; also used for other application-specific control messages that can affect other pipes.
\item\textit{Control} -- initial configuration after device plug-in; also used for other application-specific control messages that can affect other pipes.
\item\textit{Bulk} -- used for burst transfers of large messages
\item\textit{Bulk} -- used for bulk transfers of large messages
\item\textit{Isochronous} -- streaming with guaranteed low latency; designed for audio or video streams where some data loss is preferred over stuttering
\item\textit{Isochronous} -- streaming with guaranteed low latency; designed for audio or video streams where some data loss is preferred over stuttering
\item\textit{Interrupt} -- low latency short messages, used for human interface devices like mice and keyboards
\item\textit{Interrupt} -- short messages with guaranteed low latency, used for human interface devices like mice and keyboards
\end{itemize}
\end{itemize}
\subsection{Interfaces and Classes}
\subsection{Interfaces and Classes}
The function's endpoints are grouped into \textit{interfaces}. An interface describes a logical connection of endpoints, such as the reception and transmission endpoints that belong together. An interface is assigned a \textit{class} defining how it should be used.
The function's endpoints are grouped into \textit{interfaces}. An interface describes a logical relation between endpoints, such as the reception and transmission endpoints that belong together. An interface is assigned a \textit{class} defining how it should be used.
Standard classes are defined by the USB specification~\cite{usb-class-list} to provide a uniform way of interfacing devices of the same type, such as human-interface devices (mice, keyboards, gamepads) or mass storage devices. The use of standard classes makes it possible to re-use the same driver software for devices from different manufacturers.
Standard classes are defined by the \gls{USB} specification~\cite{usb-class-list} to provide a uniform way of interfacing devices of the same type, such as human interface or mass storage devices. The use of standard classes makes it possible to re-use the same software driver for devices from different manufacturers.
The class used for the GEX's ``virtual COM port'' function was originally meant for telephone modems, a common way of connecting to the Internet at the time the first versions of USB were developed. A device using this class will show as \verb|/dev/ttyACM0| on Linux and as a COM port on Windows, provided the system supports it natively or the right driver is installed.
\subsection{Descriptors}
\subsection{Descriptors}
USB devices are introspectable, that is, the host can learn about a newly connected device automatically by probing it, without any user interaction. This is accomplished using a\textit{descriptor table}, a binary structure stored in the function and read by the host through the control endpoint (default pipe) after the device is attached.
USB devices are introspectable, that is, the host can learn about a newly connected device automatically by probing it, without any user interaction. This is accomplished using the\textit{descriptor table}, a binary structure stored in the function and read by the host through the control endpoint (default pipe) after the device is attached.
Each descriptor starts with a declaration of its length (in bytes), followed by its type, allowing the host to skip unknown descriptors without having to discard the rest of the table. The descriptors are logically nested and form a tree-like structure, but they are stored sequentially in the descriptor table and the lengths do no include sub-descriptors.
Each descriptor starts with a declaration of its length (in bytes), followed by its type, allowing the host to skip unknown descriptors without having to discard the rest of the table. The descriptors are logically nested and form a tree-like structure, but they are stored sequentially in the descriptor table and the specified lengths do no include sub-descriptors.
The topmost descriptor holds information about the entire function, including the vendor and product IDs which uniquely identifies the device model. It is followed by a Configuration descriptor, grouping a set of interfaces. More than one configuration may be present and available for the host to choose from; however, this is rarely used or needed. Each configuration descriptor is followed by one or more interface descriptors, each with its class-specific sub-descriptors and/or endpoint descriptors.
The topmost descriptor holds information about the entire function, including the vendor and product IDs (VID and PID), which uniquely identifies the device model. This is followed by a Configuration descriptor, each grouping a set of interfaces. More than one configuration may be present and available for the host to choose from; however, this is rarely used or needed. The configuration descriptor is followed by one or more interface descriptors, each with its class-specific sub-descriptors and/or endpoint descriptors.
The descriptor table used by GEX is captured in \cref{fig:gex_descriptors} for illustration. The vendor and product IDs were obtained from the pid.codes repository~\cite{pidcodes} providing free product codes to open source projects. The official way of obtaining the unique code involves high recurring fees (\$4000 per annum) to the USB Implementers Forum, Inc. and is therefore not affordable for non-commercial use; alternatively, a product code may be obtained from some \gls{MCU} vendors if their product is used in the device.
The descriptor table reported by GEX is captured in \cref{fig:gex_descriptors} for illustration. The VID and PID codes were obtained from the pid.codes repository~\cite{pidcodes} providing free product codes to open-source projects. The official way of obtaining a vendor ID involves high recurring fees (\$4000 per annum) to the USB-IF, and is therefore not accessible for open-source and non-profit projects.
\newpage
\newpage
\input{fig.gex-descriptors}
\input{fig.gex-descriptors}
\section{USB Physical Layer}
\section{USB Physical Layer}
\gls{USB} uses differential signaling with \gls{NRZI} encoding and bit stuffing (the insertion of dummy bits to prevent long intervals in the same \gls{DC} level). The encoding, together with frame formatting, checksum verification, retransmission, and other low-level aspects of the \gls{USB} connection are entirely handled by the \gls{USB} physical interface block in the microcontroller's silicon. Normally we do not need to worry about those details; nonetheless, a curious reader may find more information in chapters 7 and 8 of~\cite{usbif-spec}. The electrical characteristics of the physical connection deserve more attention, as they need to be understood correctly for a successful schematic and \gls{PCB} design.
\gls{USB} uses differential signaling with the \gls{NRZI} encoding and bit stuffing (the insertion of dummy bits to prevent long intervals in the same \gls{DC} level). The encoding, together with frame formatting, checksum verification, retransmission, and other low-level aspects of the \gls{USB} connection are entirely handled by the \gls{USB} physical interface block in the microcontroller's silicon. Normally we do not need to worry about those details; nonetheless, a curious reader may find more information in chapters 7 and 8 of the specification~\cite{usbif-spec}. The electrical characteristics of the physical connection deserve more attention, as they need to be understood correctly for a successful schematic and \gls{PCB} design.
The \gls{USB} cable contains 4 conductors: V$_\mathrm{BUS}$ (+5\,V), D+, D--, and \gls{GND}. The data lines, D+ and D--, are also commonly labeled DP and DM. This differential pair should be routed in parallel on the \gls{PCB} and kept at the same length.
The \gls{USB} cable contains 4 conductors: V$_\mathrm{BUS}$ (+5\,V), D+, D--, and \gls{GND}. The data lines, D+ and D--, are also commonly labeled DP and DM. This differential pair should be routed in parallel on the \gls{PCB} and kept at the same length.
\gls{USB} versions that share the same connector are backward compatible. The desired bus speed is requested by the device using a 1.5\,k$\Omega$ pull-up resistor to 3.3\,V on one of the data lines: D+ pulled high for Full Speed (shown in \cref{fig:usb_pullup_fs}), D-- pulled high for Low Speed. The polarity of the differential signals is also inverted depending on the used speed, as the idle level changes. Some microcontrollers integrate the correct pull-up resistor inside the \gls{USB} peripheral block (including out STM32F072), removing the need for an external resistor.
\gls{USB} versions that share the same connector are backward compatible. The desired bus speed is requested by the device using a 1.5\,k$\Omega$ pull-up resistor to 3.3\,V on one of the data lines: D+ is pulled high for Full Speed (shown in \cref{fig:usb_pullup_fs}), D-- for Low Speed. The polarity of the differential signals is inverted depending on the used speed, as the idle level changes. Some microcontrollers integrate the correct pull-up resistor inside the \gls{USB} peripheral block (including out STM32F072), removing the need for an external resistor.
\begin{figure}[h]
\begin{figure}[h]
\centering
\centering
@ -70,31 +68,29 @@ The \gls{USB} cable contains 4 conductors: V$_\mathrm{BUS}$ (+5\,V), D+, D--, an
\caption[USB pull-ups]{\label{fig:usb_pullup_fs}Pull-up and pull-down resistors near the host and a Full Speed function, as prescribed by the USB specification rev. 2.0}
\caption[USB pull-ups]{\label{fig:usb_pullup_fs}Pull-up and pull-down resistors near the host and a Full Speed function, as prescribed by the USB specification rev. 2.0}
\end{figure}
\end{figure}
When a function needs to be re-enumerated by the host, which causes a reload of the descriptor table and the re-attachment of software drivers, it can momentarily remove the pull-up resistor, which the host will interpret as if the device was disconnected. With an internal pull-up, this can be done by flipping a bit in a control register. An external resistor may be connected through a transistor controlled by a \gls{GPIO} pin. As discussed in~\cite{eev-gpio-pu}, a GPIO pin might be used to drive the pull-up directly, though this has not been verified by the author.
When a function needs to be re-enumerated by the host, which causes a reload of the descriptor table and the re-attachment of software drivers, it can momentarily remove the pull-up resistor, which the host will interpret as if the device was disconnected. With an internal pull-up, this can be done by flipping a bit in a control register. An external resistor may be connected through a transistor controlled by a \gls{GPIO} pin. As discussed in~\cite{eev-gpio-pu}, a GPIO pin might be used to drive the pull-up directly, though this has not been verified by the author and is not mentioned in the specification.
The V$_\mathrm{BUS}$ line supplies power to \textit{bus-powered} devices. \textit{Self-powered} devices can leave this pin unconnected and instead use an external power supply. The maximal current drawn from the V$_\mathrm{BUS}$ line is configured using a descriptor and should not be exceeded, but experiments suggest this is often not enforced.
The V$_\mathrm{BUS}$ line supplies power to \textit{bus-powered} devices. \textit{Self-powered} devices can leave this pin unconnected and instead use an external power supply. The maximal current drawn from the V$_\mathrm{BUS}$ line is configured using a descriptor and should not be exceeded, but experiments suggest this is often not enforced. Before the descriptor table has been read, a default current rating will be used.
\noindentMore details about the electrical and physical connection may be found in~\cite{usb-nutshell}, sections \textit{Connectors} through \textit{Power}.
More details about the electrical and physical connection may be found, besides the specification, in the sections \textit{Connectors} through \textit{Power} of~\cite{usb-nutshell}.
\section{USB Classes}\label{sec:usb_classes}
\section{USB Classes}\label{sec:usb_classes}
This section explains the classes used in the GEX firmware. A list of all standard classes with a more detailed explanation can be found in~\cite{usb-class-list}.
This section explains the classes used in the GEX firmware. All standard classes and their descriptions may be found in the official list~\cite{usb-class-list}.
\subsection{Mass Storage Class}\label{sec:msc}
\subsection{Mass Storage Class}\label{sec:msc}
The \gls{MSC} is supported by all modern \gls{PC} operating systems to support \gls{USB} thumb drives, external disks, memory card readers, and other storage devices.
The \gls{MSC} is implemented by all modern \gls{PC} operating systems to support \gls{USB} thumb drives, external disks, memory card readers, and other storage devices.
The \gls{MSC} specification~\cite{usbif-msco} defines multiple \textit{transport protocols} that can be selected using the descriptors. The \gls{BOT}~\cite{usbif-bot} will be used for its simplicity. \gls{BOT} uses two bulk endpoints for reading and writing blocks of data and for the exchange of control commands and status messages.
The \gls{MSC} specification~\cite{usbif-msco} defines multiple \textit{transport protocols} that can be selected using descriptors. The \gls{BOT}~\cite{usbif-bot} will be used for its simplicity. \gls{BOT} uses two bulk endpoints for reading and writing blocks of data and for the exchange of control commands and status messages.
For the mass storage device to be recognized by the host operating system, it must also implement a \textit{command set}. Most mass storage devices use the \textit{\gls{SCSI} Transparent command set}
For the mass storage device to be recognized by the host \gls{OS}, it must also implement a \textit{command set}. Most mass storage devices use the \textit{\gls{SCSI} Transparent command set}
\footnote{To confirm this assertion, the descriptors of five thumb drives and an external hard disk were analyzed using \verb|lsusb|. All but one device used the SCSI command set, one (the oldest thumb drive) used \textit{SFF-8070i}. A list of possible command sets can be found in~\cite{usbif-msco}}.
\footnote{To confirm this assertion, the descriptors of five thumb drives and an external hard disk were analyzed using \verb|lsusb|. All but one device used the SCSI command set, one (the oldest thumb drive) used \textit{SFF-8070i}. A list of possible command sets can be found in~\cite{usbif-msco}}. Unfortunately, the \gls{SCSI} Transparent command set appears to have been deliberately left unspecified by the USB-IF (see discussion in~\cite{usb-tscsi-wtf} and the surrounding thread) and the protocol presently used under this name is an industry standard without a clear definition. Some information may be found in~\cite{usb-tscsi} and by examining the source code of the USB Device driver library provided by ST Microelectronics.
Unfortunately, the \gls{SCSI} Transparent command set appears to have been deliberately left unspecified for license or copyright reasons (see discussion in~\cite{usb-tscsi-wtf} and the surrounding thread) and the protocol presently used under this name is an industry standard without a clear definition. Some pointers may be found in~\cite{usb-tscsi} and by examining the source code of the USB Device driver library provided by ST Microelectronics.
The \gls{SCSI} Transparent command set lets the host read information about the attached storage device, such as its capacity, and check for media presence and readiness to write or detach. This is used, e.g., for the ``Safely Remove'' function, which ensures that all internal buffers have been written to the flash memory.
This command set lets the host read information about the attached storage device, such as its capacity, and check for media presence and readiness to write or detach. This is used, e.g., for the ``Safely Remove'' function, which ensures that all internal buffers have been written to the flash memory.
In order to emulate a mass storage device without having a physical storage medium, we need to generate and parse the file system on-the-fly as the host \gls{OS} tries to access it. This will be discussed in \cref{sec:fat16}.
In order to emulate a mass storage device without having a physical storage medium, we need to generate and parse the file system on-the-fly as the host \gls{OS} tries to access it. This will be discussed in \cref{sec:fat16}.
@ -102,11 +98,13 @@ In order to emulate a mass storage device without having a physical storage medi
Historically meant for modem communication, \gls{CDCACM} is now the de facto standard way of making \gls{USB} devices appear as serial ports on the host \gls{OS}. Its specification can be found in~\cite{usbif-cdc}. \gls{CDCACM} is a combination of two related classes, \gls{CDC} handling the data communication and \gls{ACM}, which defines control commands. Three endpoints are used: bulk IN, bulk OUT, and interrupt OUT.
The \acrshort{CDCACM} class, used for GEX's ``virtual COM port'' function and specified in~\cite{usbif-cdc}, was originally meant for telephone modems, a common way of connecting to the internet at the time the \gls{USB} specification was developed.
The \gls{CDCACM} is now the de facto standard way of making \gls{USB} devices appear as serial ports on the host \gls{OS}. A device using this class will show as \verb|/dev/ttyACM0| on Linux and as a COM port on Windows, provided the system supports it natively or the right driver is installed.
The interrupt endpoint is used for control commands, such as toggling the auxiliary lines of RS-232 or setting the baud rate. Since GEX does not translate the data communication to any physical UART, those commands are not applicable and can be silently ignored.
\gls{CDCACM} is a combination of two related classes, \gls{CDC} handling the data communication and \gls{ACM}, which defines control commands. Three endpoints are used: bulk IN, bulk OUT, and interrupt OUT. The interrupt endpoint delivers control commands, such as toggling the auxiliary lines of RS-232, or setting the baud rate. Since GEX does not translate the data communication to any physical UART, these commands are not applicable and can be silently ignored.
An interesting property of the \gls{CDC} class is that the bulk endpoints transport raw data without any wrapping frames. By changing the interface's class in the descriptor table to 255 (\textit{Vendor Specific Class}), we can retain the messaging functionality of the designated endpoints, while accessing the endpoints device directly using, e.g., libUSB, without any interference from the \gls{OS}. This approach is also used to hide the \gls{MSC} interface when it is not needed.
An interesting property of the \gls{CDC} class is that the bulk endpoints transport raw data without any wrapping frames. By changing the interface's class in the descriptor table to 255 (\textit{Vendor Specific Class}), we can retain the messaging functionality of the designated endpoints while accessing the endpoints device directly, without any interference from the \gls{OS}. This approach is also used to hide the \gls{MSC} interface when not needed.
@ -114,5 +112,5 @@ The original \gls{USB} specification expected that each function will have only
The \gls{IAD} is an entry in the descriptor table that defines which interfaces belong together and should be handled by the same software driver. To use the \gls{IAD}, the function's class must be set to 0xEF, subclass 0x02, and protocol 0x01 in the top level descriptor, so that the \gls{OS} knows to look for this descriptor before binding drivers to any interfaces.
The \gls{IAD} is an entry in the descriptor table that defines which interfaces belong together and should be handled by the same software driver. To use the \gls{IAD}, the function's class must be set to 0xEF, subclass 0x02, and protocol 0x01 in the top level descriptor, so that the \gls{OS} knows to look for this descriptor before binding drivers to any interfaces.
In GEX, the \gls{IAD} is used to tie together the \gls{CDC} and \gls{ACM} interfaces while leaving out the \gls{MSC} interface which should be handled by a different driver. To make this work, a new \textit{composite class} was created as a wrapper for the library-provided \gls{MSC} and \gls{CDCACM} implementation.
In GEX, the \gls{IAD} is used to tie together the \gls{CDC} and \gls{ACM} interfaces while leaving out the \gls{MSC} interface which should be handled by a different driver. To make this work, a new \textit{composite class} was created in the source code, as a wrapper for the library-provided \gls{MSC} and \gls{CDCACM} implementation. The composite class handles data routing to the individual sub-classes and provides custom descriptors with the \gls{IAD}.
Four methods of a wireless connection have been considered: Bluetooth (perhaps with the Texas Instruments CC2541), WiFi with the Espressif ESP8266, a 868\,MHz long range radio link with the Semtech SX1276, and a 2.4\,GHz radio link with the nRF24L01+. Bluetooth was dismissed early for its complexity, and the ESP8266 for its high power consumption, although both solutions might be viable for certain applications and with more development time.
Four methods of a wireless connection were considered: Bluetooth (perhaps with the Texas Instruments CC2541), WiFi with the Espressif ESP8266, a 868\,MHz long range radio link with the Semtech SX1276, and a 2.4\,GHz radio link with the nRF24L01+. Bluetooth was dismissed early for its complexity, and the ESP8266 for its high power consumption, although both solutions might be viable for certain applications and with more development time.
\begin{figure}[h]
\begin{figure}[h]
\centering
\centering
@ -10,7 +10,7 @@ Four methods of a wireless connection have been considered: Bluetooth (perhaps w
\section{Modulations Overview}
\section{Modulations Overview}
A brief overview of the different signal modulation techniques is presented here to aid the reader with understanding of \cref{fig:nrf_sx_comparison} and the rest of the chapter.
A brief overview of the different signal modulation techniques is presented here to aid the reader with the understanding of \cref{fig:nrf_sx_comparison} and the rest of the chapter.
\subsection{On-Off Keying (OOK)}
\subsection{On-Off Keying (OOK)}
@ -75,7 +75,7 @@ Both devices implement some form of a packet engine with error checking; that of
\section{Wireless Link with the nRF24L01+}
\section{Wireless Link with the nRF24L01+}
The nRF24L01+ was selected to be integrated into GEX thanks to its inclusion of the ShockBurst engine, higher possible data rates and significantly lower price. The SX1276, nonetheless, remains an interesting option, should the need for a long range communication arise.
We chose the nRF24L01+ to be integrated into GEX for its inclusion of the ShockBurst engine, higher possible data rates, and significantly lower price. The SX1276, nonetheless, remains an interesting option, should the need for a long range communication arise.
A pair of these radio modules can form a bidirectional data connection, functionally replacing \gls{USB} or \gls{UART} as a communication interface. However, we need to connect the second module to the \gls{PC} to control GEX through the radio link. A separate \gls{USB} device, a \textit{wireless gateway}, was developed for this purpose; its hardware will be presented in \cref{sec:rfgateway}.
A pair of these radio modules can form a bidirectional data connection, functionally replacing \gls{USB} or \gls{UART} as a communication interface. However, we need to connect the second module to the \gls{PC} to control GEX through the radio link. A separate \gls{USB} device, a \textit{wireless gateway}, was developed for this purpose; its hardware will be presented in \cref{sec:rfgateway}.
Before you lies the work of many long days and sleepless nights, the result of years of study and planning. This work is released to the commons as a payback for the many open-source projects that allowed it to become what it has.
Before you lies the work of many long days and sleepless nights, the result of years of study and planning. This work is released to the commons as payback for the many open-source projects that allowed it to become what it has.
\vspace{5pt}
\vspace{5pt}
I would like to thank my tutors for the knowledge, and to my friends for the encouragement to pull this off and create something I can be proud of.
I would like to thank my tutors for the knowledge, and my friends for the encouragement to pull this off and create something I can be proud of.
