added more documentation

7 years ago · d36df8372a
parent 2d188b935a
commit d36df8372a
6 changed files with 311 additions and 80 deletions
--- a/01_CORE.md
+++ b/01_CORE.md
@ -1,80 +0,0 @@
 # GEX core protocol
 ## Framing layer
 GEX uses [TinyFrame](https://github.com/MightyPork/TinyFrame) to form and parse data frames.
 - Frames are sent and received using two USB bulk endpoints (VCOM or raw access) or through UART.
 - Transmitted frame length is virtually unlimited
 - Received frame length is limited by the Rx buffer size.
 ### Frame structure
 ```none
 ,------+----------+---------+------+------------+- - - - -+------------,                
 | SOF  | frame_id | pld_len | type | head_cksum | payload | pld_cksum  |                
 | 1    | 2        | 2       | 1    | 1          | ...     | 1          | <- size (bytes)
 '------+----------+---------+------+------------+- - - - -+------------'      
 ```
 - SOF byte is `0x01`
 - Checksum = inverted XOR of all bytes
 The checksum is used to catch implementation mistakes (such as mistaken CR-LF transformations).
 USB bulk transfers have CRC error checking built-in.
 *Frame ID* is incremented with each transaction (message, request-response or longer). 
 The highest ID bit identifies the peer who started the transaction.
 *Type* defines the meaning of the frame. A list is attached below.
 ## Message types
 ### General messages
 - `0x00` ... `SUCCESS` - Generic success response; used by default in all responses; payload is transaction-specific
 - `0x01` ... `PING` - Ping request, used to test connection
  - Response: ASCII string with the version and platform name
 - `0x02` ... `ERROR` - Generic failure response (when a request fails to execute)
  - Response: ASCII string with the error message
 ### Bulk transfer (large multi-frame transfer)
 Bulk transfer is used for reading / writing large files that exceed the TinyFrame buffer sizes.
 - `0x03` ... `BULK_READ_OFFER` - Offer of data to read. 
  - Payload: u32 total len
 - `0x04` ... `BULK_READ_POLL` - Request to read a previously announced chunk (`BULK_READ_OFFER`)
  - Payload: u32 max chunk size to read
  - Response: a `BULK_DATA` frame
 - `0x05` ... `BULK_WRITE_OFFER` - Offer to receive data in a write transaction.
  - This is an affirmation to some other frame that introduced a request to write bulk data.
  - Payload: u32 max total size, u32 max chunk size
 - `0x06` ... `BULK_DATA` - Writing a chunk, or sending a chunk to master.
 - `0x07` ... `BULK_END` - Bulk transfer is done, no more data to read or write. Recipient shall check total len and discard the received data on mismatch. 
 - `0x08` ... `BULK_ABORT` - Discard the ongoing transfer (sent by either peer)
 ### Unit messages
 Units are instances of a particular unit driver. A unit driver is e.g. Digital Output or SPI.
 Each unit has a TYPE (which driver to use), NAME (for user convenience) and CALLSIGN (for messages)
 In the INI file, all three are shown in the unit section: `[type:name@callsign]`
 Units can accept commands and generate events. Both are identified by 1-byte codes.
 - `0x10` ... `UNIT_REQUEST` - Command addressed to a particular unit
  - Payload: u8 callsign, u8 command, rest - data for a unit driver
 - `0x11` ... `UNIT_REPORT` - Spontaneous report from a unit
  - Payload: u8 callsign, u8 type, u64 usec timestamp, rest: data from unit driver
 ### System messages
 Settings management etc.
 - `0x20` ... `LIST_UNITS` - Get all active unit call-signs, types and names
  - Response: u8 count, { u8 callsign, cstring type, cstring name }
 - `0x21` ... `INI_READ` - Read the ini file via bulk
  - starts a bulk read transfer
  - Response: a `BULK_READ_OFFER` frame
 - `0x22` ... `INI_WRITE` - Write the ini file via bulk
  - starts a bulk write transfer
  - Response: a `BULK_WRITE_OFFER` frame
 - `0x23` ... `PERSIST_CFG` - Write current settings to Flash (the equivalent of replacing the lock jumper / pushing the button if VFS is active)
--- a/FRAME_FORMAT.md
+++ b/FRAME_FORMAT.md
@ -0,0 +1,111 @@
 # GEX's low-level communication protocol
 ## Framing layer
 GEX uses [TinyFrame](https://github.com/MightyPork/TinyFrame) to form and parse data frames.
 - Frames are normally sent and received using two USB bulk endpoints (VCOM or raw access) or through UART.
 - Transmitted frame length is virtually unlimited
 - Received frame length is limited by the Rx buffer size. This is overcome using a Bulk Write function.
 ### Frame structure
 See the TinyFrame documentation for detailed description of the frame structure and protocol functions.
 Here's the configuration used by GEX:
 ```none
 ,------+----------+---------+------+------------+- - - - -+------------,
 | SOF  | frame_id | pld_len | type | head_cksum | payload | pld_cksum  |
 | 1    | 2        | 2       | 1    | 1          | ...     | 1          | <- size (bytes)
 '------+----------+---------+------+------------+- - - - -+------------'
 ```
 - SOF byte is `0x01`
 - Checksum = inverted XOR of all bytes
 USB bulk transfers have CRC error checking built-in. The checksum field is mainly used to catch
 implementation mistakes, such as a CR-LF transformation.
 *Frame ID* is incremented with each transaction (message, request/response, or longer).
 The highest ID bit identifies the peer who started the transaction ("master", "slave").
 *Type* defines the meaning of the frame within a higher level protocol. A list of all types is attached below.
 ## Message types
 ### General messages
 - `0x00` ... `SUCCESS` - Generic success response; used by default in all responses;
  - *Payload:*
    - optional, differs based on the request
 - `0x01` ... `PING` - Ping request, used to test connection
  - *Response:*
    - `SUCCESS` frame with an ASCII string containing the GEX version and the hardware platform name
 - `0x02` ... `ERROR` - Generic failure response, used when a request fails to execute
  - *Payload:*
    - ASCII string with the error message
 ### Bulk transfer (large multi-frame transfer)
 Bulk transfer is used for reading / writing large files that exceed the TinyFrame buffer sizes.
 - `0x03` ... `BULK_READ_OFFER` - Offer of data to read.
  - *Payload:*
    - u32 - total len
 - `0x04` ... `BULK_READ_POLL` - Request to read a previously announced chunk (`BULK_READ_OFFER`)
  - *Payload:*
    - u32 - max chunk size to read
  - Response: a `BULK_DATA` frame (see below)
 - `0x05` ... `BULK_WRITE_OFFER` - Offer to receive data in a write transaction.
  - This is a report of readiness after some other frame introduced a request to write bulk data
    (e.g. writing a config file)
  - *Payload:*
    - u32 - max total size
    - u32 - max chunk size
 - `0x06` ... `BULK_DATA` - Writing a chunk, or sending a chunk to master.
  - *Payload:*
    - a block of data
 - `0x07` ... `BULK_END` - Bulk transfer is done, no more data to read or write. Recipient shall check total length and discard the received data on mismatch.
 - `0x08` ... `BULK_ABORT` - Abort and discard the ongoing transfer (sent by either peer)
 ### Unit messages
 Units are instances of a particular unit driver. A unit driver is e.g. Digital Output or SPI.
 Each unit has a TYPE (which driver to use), NAME (for user convenience) and CALLSIGN (for messages)
 In the INI file, all three are shown in the unit section: `[type:name@callsign]`
 Units can accept commands and generate events. Both are identified by 1-byte codes.
 - `0x10` ... `UNIT_REQUEST` - Command addressed to a particular unit
  - *Payload:*
    - u8 callsign
    - u8 command
    - rest - data for a unit driver
 - `0x11` ... `UNIT_REPORT` - Spontaneous report from a unit
  - *Payload:*
    - u8 callsign
    - u8 report type
    - u64 usec timestamp
    - rest - data from unit driver
 ### System messages
 Settings management etc.
 - `0x20` ... `LIST_UNITS` - Get all active unit call-signs, types and names
  - *Response:*
    - u8 number of units
    - repeat for each unit:
      - u8 callsign
      - cstring type
      - cstring name
 - `0x21` ... `INI_READ` - Read the ini file via bulk
  - starts a bulk read transfer
  - *Response:*
    - a `BULK_READ_OFFER` frame (see above)
 - `0x22` ... `INI_WRITE` - Write the ini file via bulk
  - starts a bulk write transfer
  - *Response:*
    - a `BULK_WRITE_OFFER` frame (see above)
 - `0x23` ... `PERSIST_CFG` - Write current settings to Flash
  - This is the equivalent of replacing the lock jumper / pushing the button when the virtual filesystem is enabled.
  - If the virtual filesystem is enabled, this also closes it.
--- a/INI_FILES.md
+++ b/INI_FILES.md
@ -0,0 +1,97 @@
 # GEX's config files
 GEX's configuration is stored in the internal flash memory in a packed binary form.
 To make its editing easier for the user, the configuration is accessible as INI files
 annotated by numerous comments explaining the individual config options.
 ## Filesystem access
 There are two config files: `UNITS.INI` and `SYSTEM.INI`.
 To access those files, the user presses the LOCK button (or removes the LOCK jumper),
 which enables an emulated FAT16 storage that will appear on the host computer as a USB disk.
 Some hardware platforms require that the jumper be removed before plugging in the USB cable,
 as it can't trigger re-enumeration without external circuitry, which is missing on those
 evaluation boards.
 It's recommended to copy the INI files to a real disk before editing, as some editors create
 temporary back-up files that can confuse the filesystem emulator. However, in-place editing
 is also possible with some editors.
 *This method is the only way of accessing the `SYSTEM.INI` file, which can reconfigure
 the main communication port.*
 After writing the changed files back to the virtual disk, it will briefly disappear
 and an updated version of the files will become available for review.
 This is when you can check for error messages.
 To confirm the config modifications (they take effect imemdiately, but are not written to flash yet),
 push the LOCK button again (or replace the LOCK jumper).
 ## API access
 The `UNITS.INI` file may be accessed through the communication port.
 The file is read using Bulk Read (see [FRAME_FORMAT.md](FRAME_FORMAT.md)) after sending a `INI_READ` request.
 The modified file is written back using Bulk Write after sending a `INI_WRITE` request.
 After writing the changed configuration, it may be persisted to flash using a `PERSIST_CFG` command.
 This is not required if the new configuration will be used only temporarily; the original settings
 will be restored after a restart.
 ## UNITS.INI file structure
 The `UNITS.INI` file is interactive. The general layout is as follows:
 ```ini
 ## UNITS.INI
 ## GEX v0.0.1 on STM32F072-HUB
 ## built Mar 17 2018 at 17:53:15
 # Overwrite this file to change settings.
 # Press the LOCK button to save them to Flash.
 [UNITS]
 # Create units by adding their names next to a type (e.g. DO=A,B),
 # remove the same way. Reload to update the unit sections below.
 # Digital output
 DO=
 # Digital input with triggers
 DI=btn,btn2
 # (... more unit types here)
 [DO:btn@1]
 # Port name
 port=B
 # Pins (comma separated, supports ranges)
 pins=2
 # Initially high pins
 initial=2
 # Open-drain pins
 open-drain=
 ```
 - The keys in the `[UNITS]` section are lists of instances for each unit type.
  After adding a name to the list and saving the file, a new unit section will appear below, offering to configure
  the new unit as needed.
 - Each unit section starts with `[type:name@callsign]`.The name must match the name used in the unit lists above.
  The callsign can be freely changed, as long as there is no conflict. Callsign 0 is forbidden and will be changed
  to a new unique callsign on save.
 - If an error occurs during unit initialization, an error message will appear under the unit's section header.
 ## GEX reconfiguration workflow
 1. Enable the mass storage using the LOCK button or jumper (or use the API for reading and writing the file)
 2. Open the `UNITS.INI` file in a text editor
 3. If adding or removing unit instances, add their names to the right type (eg. `DO=LED`), save and reload the file.
   On some systems, it's necessary to close the editor right after saving and then reopen it again after the disk
   refreshes.
 4. Configure the units as needed; adjust callsigns if desired. Save and reload the file.
 5. Check for error messages under unit headers. Correct any problems. Save and reload to apply any changes.
 6. Push the LOCK button or replace the jumper to persist all changes to flash. If using the API, issue
   the `PERSIST_CFG` command for the same effect.
 This is much easier when done through the API, perhaps using the PyQt GEX configuration utility included as an
 example in the Python client repository.
--- a/INTERNAL_STRUCTURE.md
+++ b/INTERNAL_STRUCTURE.md
@ -0,0 +1,44 @@
 # GEX's internal structure
 GEX is modular, composed of functional blocks called *units*. A unit can be e.g. SPI or a digital output.
 There can be more than one instance of a given unit, based on the microcontroller's hardware resources.
 Some units use hardware peripherals, others are implemented in software. Those "soft" units (typically
 bit-banged protocols) are limited only by the available GPIO pins.
 ## Resources
 A system of resource claiming is implemented to avoid units conflicting with each other and interfering with
 peripherals used internally by the system (such as the LED pin). Each unit claims the resources it uses during
 initialization. If the resource needed is not available, it may try to claim an alternate resource or fail to
 initialize.
 A init failure is reported in the config file as an error message under the unit's header.
 An example of defined resources:
 - all GPIO pins
 - timers/counters
 - SPI, I2C, USART peripherals
 - DMA channels
 ## Unit identification
 Each unit has a *name* and a *callsign* (1-255) used for message addressing. Both are defined in the config file.
 There can be up to 255 units, in practice fewer due to storage limitations.
 ## Unit commands and events
 Units can receive command messages (`UNIT_REQUEST`) and report asynchronous events using `UNIT_REPORT` frames.
 A command is identified by a number 0-127. The highest bit (0x80) of the command number is reserved to request
 confirmation (a `SUCCESS` frame) if the command doesn't normally generate a response. This is used by the client
 library to check that an operation was completed correctly (as opposed to e.g. the firmware crashing and rebooting
 through the watchdog).
 Unit events include the unit's callsign, a number 0-255 identifying the event type, and a 64-bit microsecond
 timestamp.
 See [FRAME_FORMAT.md](FRAME_FORMAT.md) for the payloads structure.
--- a/README.md
+++ b/README.md
@ -1,3 +1,10 @@
 # GEX protocol documentation
 This repository specifies the control protocol implemented by [gex-core](https://github.com/gexpander/gex-core).
 - [FRAME_FORMAT.md](FRAME_FORMAT.md) - Low level frame format, message types
 - [INTERNAL_STRUCTURE.md](INTERNAL_STRUCTURE.md) - GEX's internal structure, units, requests and events
 - [INI_FILES.md](INI_FILES.md) - INI config files, filesystem and API access
 - [UNIT_DO.md](UNIT_DO.md) - Digital Output unit
--- a/UNIT_DO.md
+++ b/UNIT_DO.md
@ -0,0 +1,52 @@
 # Digital Output
 - Direct output access to one or more pins of a port.
 - All selected pins are written simultaneously.
 - Supports generating output pulses of precise length.
 Pins are ordered in a descending order (DOWNTO) and accessed in a packed format,
 e.g. if pins 1 and 4 are selected (0b10010 on the port), the control word has
 two bits (4)(1) and e.g. 0b10 means pin 4, 0b11 both. This makes accessing the
 block of pins easier, e.g. when using them to drive a parallel bus.
 ## Commands
 ### WRITE
 Write a value to all defined pins.
 *Payload:*
 - u16 - new value (packed)
 ### SET
 Set pins high
 *Payload:*
 - u16 - pins to set high (packed)
 ### CLEAR
 Set pins low
 *Payload:*
 - u16 - pins to set low (packed)
 ### TOGGLE
 Toggle selected pins (high - low)
 *Payload:*
 - u16 - pins to toggle (packed)
 ### PULSE
 Send a pulse. The start will be aligned to 1 us or 1 ms (based on pulse length) of the internal timebase to ensure the highest length precision. (This alignment reduces time jitter.)
 *Payload:*
 - u16 - pins to generate the pulse on (packed)
 - u8 - pulse active level (0, 1)
 - u8 - range (0 - milliseconds, 1 - microseconds)
 - u16 - duration in the selected range
 *NOTE:* The microsecond range supports durations only up to 999 (1 ms), higher
 numbers will be divided by 1000 and use the millisecond range instead.
 ## Events
 *No events defined for this unit type.*