lib: Add workaround for Linux usbfs mmap() bug

The Linux Kernel has a bug on ARM/ARM64 systems where the USB CMA
memory is incorrectly mapped to userspace, breaking zerocopy.

When the Kernel allocates the memory, it clears it with memset().
If the mapping worked correctly, we should have zeroed out buffers,
if it doesn't, we get random Kernel memory. We now check for this,
and fall back to buffers in userspace if that's the case.

Signed-off-by: Steve Markgraf <steve@steve-m.de>

src/librtlsdr.c

@@ -1755,11 +1755,26 @@ static int _rtlsdr_alloc_async_buffers(rtlsdr_dev_t *dev)
 	for (i = 0; i < dev->xfer_buf_num; ++i) {
 		dev->xfer_buf[i] = libusb_dev_mem_alloc(dev->devh, dev->xfer_buf_len);
 
-		if (!dev->xfer_buf[i]) {
+		if (dev->xfer_buf[i]) {
+			/* Check if Kernel usbfs mmap() bug is present: if the
+			 * mapping is correct, the buffers point to memory that
+			 * was memset to 0 by the Kernel, otherwise, they point
+			 * to random memory. We check if the buffers are zeroed
+			 * and otherwise fall back to buffers in userspace.
+			 */
+			if (dev->xfer_buf[i][0] || memcmp(dev->xfer_buf[i],
+							  dev->xfer_buf[i] + 1,
+							  dev->xfer_buf_len - 1)) {
+				fprintf(stderr, "Detected Kernel usbfs mmap() "
+						"bug, falling back to buffers "
+						"in userspace\n");
+				dev->use_zerocopy = 0;
+				break;
+			}
+		} else {
 			fprintf(stderr, "Failed to allocate zero-copy "
 					"buffer for transfer %d\nFalling "
 					"back to buffers in userspace\n", i);
-
 			dev->use_zerocopy = 0;
 			break;
 		}