MightyPork/tangara-fw
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Use libcppbor for much much nicer db encoding

Browse Source
This commit is contained in:
jacqueline
2023-10-13 15:05:49 +11:00
parent 20d1c280a7
commit afbf3c31f4
14 changed files with 2699 additions and 366 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/libcppbor/CMakeLists.txt
+7
View File
@@ -0,0 +1,7 @@
# Copyright 2023 jacqueline <me@jacqueline.id.au>
#
# SPDX-License-Identifier: GPL-3.0-only
idf_component_register(
SRCS cppbor.cpp cppbor_parse.cpp
INCLUDE_DIRS "include/cppbor"
)
lib/libcppbor/LICENSE
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
lib/libcppbor/cppbor.cpp
+599
View File
@@ -0,0 +1,599 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include "cppbor.h"
#include <inttypes.h>
#include <cstdint>
#include "cppbor_parse.h"
using std::string;
using std::vector;
#define CHECK(x) (void)(x)
namespace cppbor {
namespace {
template <typename T, typename Iterator, typename = std::enable_if<std::is_unsigned<T>::value>>
Iterator writeBigEndian(T value, Iterator pos) {
for (unsigned i = 0; i < sizeof(value); ++i) {
*pos++ = static_cast<uint8_t>(value >> (8 * (sizeof(value) - 1)));
value = static_cast<T>(value << 8);
}
return pos;
}
template <typename T, typename = std::enable_if<std::is_unsigned<T>::value>>
void writeBigEndian(T value, std::function<void(uint8_t)>& cb) {
for (unsigned i = 0; i < sizeof(value); ++i) {
cb(static_cast<uint8_t>(value >> (8 * (sizeof(value) - 1))));
value = static_cast<T>(value << 8);
}
}
bool cborAreAllElementsNonCompound(const Item* compoundItem) {
if (compoundItem->type() == ARRAY) {
const Array* array = compoundItem->asArray();
for (size_t n = 0; n < array->size(); n++) {
const Item* entry = (*array)[n].get();
switch (entry->type()) {
case ARRAY:
case MAP:
return false;
default:
break;
}
}
} else {
const Map* map = compoundItem->asMap();
for (auto& [keyEntry, valueEntry] : *map) {
switch (keyEntry->type()) {
case ARRAY:
case MAP:
return false;
default:
break;
}
switch (valueEntry->type()) {
case ARRAY:
case MAP:
return false;
default:
break;
}
}
}
return true;
}
bool prettyPrintInternal(const Item* item, string& out, size_t indent, size_t maxBStrSize,
const vector<string>& mapKeysToNotPrint) {
if (!item) {
out.append("<NULL>");
return false;
}
char buf[80];
string indentString(indent, ' ');
size_t tagCount = item->semanticTagCount();
while (tagCount > 0) {
--tagCount;
snprintf(buf, sizeof(buf), "tag %" PRIu64 " ", item->semanticTag(tagCount));
out.append(buf);
}
switch (item->type()) {
case SEMANTIC:
// Handled above.
break;
case UINT:
snprintf(buf, sizeof(buf), "%" PRIu64, item->asUint()->unsignedValue());
out.append(buf);
break;
case NINT:
snprintf(buf, sizeof(buf), "%" PRId64, item->asNint()->value());
out.append(buf);
break;
case BSTR: {
const uint8_t* valueData;
size_t valueSize;
const Bstr* bstr = item->asBstr();
if (bstr != nullptr) {
const vector<uint8_t>& value = bstr->value();
valueData = value.data();
valueSize = value.size();
} else {
const ViewBstr* viewBstr = item->asViewBstr();
assert(viewBstr != nullptr);
valueData = viewBstr->view().data();
valueSize = viewBstr->view().size();
}
if (valueSize > maxBStrSize) {
snprintf(buf, sizeof(buf), "<bstr size=%zd>", valueSize);
out.append(buf);
} else {
out.append("{");
for (size_t n = 0; n < valueSize; n++) {
if (n > 0) {
out.append(", ");
}
snprintf(buf, sizeof(buf), "0x%02x", valueData[n]);
out.append(buf);
}
out.append("}");
}
} break;
case TSTR:
out.append("'");
{
// TODO: escape "'" characters
if (item->asTstr() != nullptr) {
out.append(item->asTstr()->value().c_str());
} else {
const ViewTstr* viewTstr = item->asViewTstr();
assert(viewTstr != nullptr);
out.append(viewTstr->view());
}
}
out.append("'");
break;
case ARRAY: {
const Array* array = item->asArray();
if (array->size() == 0) {
out.append("[]");
} else if (cborAreAllElementsNonCompound(array)) {
out.append("[");
for (size_t n = 0; n < array->size(); n++) {
if (!prettyPrintInternal((*array)[n].get(), out, indent + 2, maxBStrSize,
mapKeysToNotPrint)) {
return false;
}
out.append(", ");
}
out.append("]");
} else {
out.append("[\n" + indentString);
for (size_t n = 0; n < array->size(); n++) {
out.append(" ");
if (!prettyPrintInternal((*array)[n].get(), out, indent + 2, maxBStrSize,
mapKeysToNotPrint)) {
return false;
}
out.append(",\n" + indentString);
}
out.append("]");
}
} break;
case MAP: {
const Map* map = item->asMap();
if (map->size() == 0) {
out.append("{}");
} else {
out.append("{\n" + indentString);
for (auto& [map_key, map_value] : *map) {
out.append(" ");
if (!prettyPrintInternal(map_key.get(), out, indent + 2, maxBStrSize,
mapKeysToNotPrint)) {
return false;
}
out.append(" : ");
if (map_key->type() == TSTR &&
std::find(mapKeysToNotPrint.begin(), mapKeysToNotPrint.end(),
map_key->asTstr()->value()) != mapKeysToNotPrint.end()) {
out.append("<not printed>");
} else {
if (!prettyPrintInternal(map_value.get(), out, indent + 2, maxBStrSize,
mapKeysToNotPrint)) {
return false;
}
}
out.append(",\n" + indentString);
}
out.append("}");
}
} break;
case SIMPLE:
const Bool* asBool = item->asSimple()->asBool();
const Null* asNull = item->asSimple()->asNull();
if (asBool != nullptr) {
out.append(asBool->value() ? "true" : "false");
} else if (asNull != nullptr) {
out.append("null");
} else {
return false;
}
break;
}
return true;
}
} // namespace
size_t headerSize(uint64_t addlInfo) {
if (addlInfo < ONE_BYTE_LENGTH) return 1;
if (addlInfo <= std::numeric_limits<uint8_t>::max()) return 2;
if (addlInfo <= std::numeric_limits<uint16_t>::max()) return 3;
if (addlInfo <= std::numeric_limits<uint32_t>::max()) return 5;
return 9;
}
uint8_t* encodeHeader(MajorType type, uint64_t addlInfo, uint8_t* pos, const uint8_t* end) {
size_t sz = headerSize(addlInfo);
if (end - pos < static_cast<ssize_t>(sz)) return nullptr;
switch (sz) {
case 1:
*pos++ = type | static_cast<uint8_t>(addlInfo);
return pos;
case 2:
*pos++ = type | ONE_BYTE_LENGTH;
*pos++ = static_cast<uint8_t>(addlInfo);
return pos;
case 3:
*pos++ = type | TWO_BYTE_LENGTH;
return writeBigEndian(static_cast<uint16_t>(addlInfo), pos);
case 5:
*pos++ = type | FOUR_BYTE_LENGTH;
return writeBigEndian(static_cast<uint32_t>(addlInfo), pos);
case 9:
*pos++ = type | EIGHT_BYTE_LENGTH;
return writeBigEndian(addlInfo, pos);
default:
CHECK(false); // Impossible to get here.
return nullptr;
}
}
void encodeHeader(MajorType type, uint64_t addlInfo, EncodeCallback encodeCallback) {
size_t sz = headerSize(addlInfo);
switch (sz) {
case 1:
encodeCallback(type | static_cast<uint8_t>(addlInfo));
break;
case 2:
encodeCallback(type | ONE_BYTE_LENGTH);
encodeCallback(static_cast<uint8_t>(addlInfo));
break;
case 3:
encodeCallback(type | TWO_BYTE_LENGTH);
writeBigEndian(static_cast<uint16_t>(addlInfo), encodeCallback);
break;
case 5:
encodeCallback(type | FOUR_BYTE_LENGTH);
writeBigEndian(static_cast<uint32_t>(addlInfo), encodeCallback);
break;
case 9:
encodeCallback(type | EIGHT_BYTE_LENGTH);
writeBigEndian(addlInfo, encodeCallback);
break;
default:
CHECK(false); // Impossible to get here.
}
}
bool Item::operator==(const Item& other) const& {
if (type() != other.type()) return false;
switch (type()) {
case UINT:
return *asUint() == *(other.asUint());
case NINT:
return *asNint() == *(other.asNint());
case BSTR:
if (asBstr() != nullptr && other.asBstr() != nullptr) {
return *asBstr() == *(other.asBstr());
}
if (asViewBstr() != nullptr && other.asViewBstr() != nullptr) {
return *asViewBstr() == *(other.asViewBstr());
}
// Interesting corner case: comparing a Bstr and ViewBstr with
// identical contents. The function currently returns false for
// this case.
// TODO: if it should return true, this needs a deep comparison
return false;
case TSTR:
if (asTstr() != nullptr && other.asTstr() != nullptr) {
return *asTstr() == *(other.asTstr());
}
if (asViewTstr() != nullptr && other.asViewTstr() != nullptr) {
return *asViewTstr() == *(other.asViewTstr());
}
// Same corner case as Bstr
return false;
case ARRAY:
return *asArray() == *(other.asArray());
case MAP:
return *asMap() == *(other.asMap());
case SIMPLE:
return *asSimple() == *(other.asSimple());
case SEMANTIC:
return *asSemanticTag() == *(other.asSemanticTag());
default:
CHECK(false); // Impossible to get here.
return false;
}
}
Nint::Nint(int64_t v) : mValue(v) {
CHECK(v < 0);
}
bool Simple::operator==(const Simple& other) const& {
if (simpleType() != other.simpleType()) return false;
switch (simpleType()) {
case BOOLEAN:
return *asBool() == *(other.asBool());
case NULL_T:
return true;
default:
CHECK(false); // Impossible to get here.
return false;
}
}
uint8_t* Bstr::encode(uint8_t* pos, const uint8_t* end) const {
pos = encodeHeader(mValue.size(), pos, end);
if (!pos || end - pos < static_cast<ptrdiff_t>(mValue.size())) return nullptr;
return std::copy(mValue.begin(), mValue.end(), pos);
}
void Bstr::encodeValue(EncodeCallback encodeCallback) const {
for (auto c : mValue) {
encodeCallback(c);
}
}
uint8_t* ViewBstr::encode(uint8_t* pos, const uint8_t* end) const {
pos = encodeHeader(mView.size(), pos, end);
if (!pos || end - pos < static_cast<ptrdiff_t>(mView.size())) return nullptr;
return std::copy(mView.begin(), mView.end(), pos);
}
void ViewBstr::encodeValue(EncodeCallback encodeCallback) const {
for (auto c : mView) {
encodeCallback(static_cast<uint8_t>(c));
}
}
uint8_t* Tstr::encode(uint8_t* pos, const uint8_t* end) const {
pos = encodeHeader(mValue.size(), pos, end);
if (!pos || end - pos < static_cast<ptrdiff_t>(mValue.size())) return nullptr;
return std::copy(mValue.begin(), mValue.end(), pos);
}
void Tstr::encodeValue(EncodeCallback encodeCallback) const {
for (auto c : mValue) {
encodeCallback(static_cast<uint8_t>(c));
}
}
uint8_t* ViewTstr::encode(uint8_t* pos, const uint8_t* end) const {
pos = encodeHeader(mView.size(), pos, end);
if (!pos || end - pos < static_cast<ptrdiff_t>(mView.size())) return nullptr;
return std::copy(mView.begin(), mView.end(), pos);
}
void ViewTstr::encodeValue(EncodeCallback encodeCallback) const {
for (auto c : mView) {
encodeCallback(static_cast<uint8_t>(c));
}
}
bool Array::operator==(const Array& other) const& {
return size() == other.size()
// Can't use vector::operator== because the contents are pointers. std::equal lets us
// provide a predicate that does the dereferencing.
&& std::equal(mEntries.begin(), mEntries.end(), other.mEntries.begin(),
[](auto& a, auto& b) -> bool { return *a == *b; });
}
uint8_t* Array::encode(uint8_t* pos, const uint8_t* end) const {
pos = encodeHeader(size(), pos, end);
if (!pos) return nullptr;
for (auto& entry : mEntries) {
pos = entry->encode(pos, end);
if (!pos) return nullptr;
}
return pos;
}
void Array::encode(EncodeCallback encodeCallback) const {
encodeHeader(size(), encodeCallback);
for (auto& entry : mEntries) {
entry->encode(encodeCallback);
}
}
std::unique_ptr<Item> Array::clone() const {
auto res = std::make_unique<Array>();
for (size_t i = 0; i < mEntries.size(); i++) {
res->add(mEntries[i]->clone());
}
return res;
}
bool Map::operator==(const Map& other) const& {
return size() == other.size()
// Can't use vector::operator== because the contents are pairs of pointers. std::equal
// lets us provide a predicate that does the dereferencing.
&& std::equal(begin(), end(), other.begin(), [](auto& a, auto& b) {
return *a.first == *b.first && *a.second == *b.second;
});
}
uint8_t* Map::encode(uint8_t* pos, const uint8_t* end) const {
pos = encodeHeader(size(), pos, end);
if (!pos) return nullptr;
for (auto& entry : mEntries) {
pos = entry.first->encode(pos, end);
if (!pos) return nullptr;
pos = entry.second->encode(pos, end);
if (!pos) return nullptr;
}
return pos;
}
void Map::encode(EncodeCallback encodeCallback) const {
encodeHeader(size(), encodeCallback);
for (auto& entry : mEntries) {
entry.first->encode(encodeCallback);
entry.second->encode(encodeCallback);
}
}
bool Map::keyLess(const Item* a, const Item* b) {
// CBOR map canonicalization rules are:
// 1. If two keys have different lengths, the shorter one sorts earlier.
if (a->encodedSize() < b->encodedSize()) return true;
if (a->encodedSize() > b->encodedSize()) return false;
// 2. If two keys have the same length, the one with the lower value in (byte-wise) lexical
// order sorts earlier. This requires encoding both items.
auto encodedA = a->encode();
auto encodedB = b->encode();
return std::lexicographical_compare(encodedA.begin(), encodedA.end(), //
encodedB.begin(), encodedB.end());
}
void recursivelyCanonicalize(std::unique_ptr<Item>& item) {
switch (item->type()) {
case UINT:
case NINT:
case BSTR:
case TSTR:
case SIMPLE:
return;
case ARRAY:
std::for_each(item->asArray()->begin(), item->asArray()->end(),
recursivelyCanonicalize);
return;
case MAP:
item->asMap()->canonicalize(true /* recurse */);
return;
case SEMANTIC:
// This can't happen. SemanticTags delegate their type() method to the contained Item's
// type.
assert(false);
return;
}
}
Map& Map::canonicalize(bool recurse) & {
if (recurse) {
for (auto& entry : mEntries) {
recursivelyCanonicalize(entry.first);
recursivelyCanonicalize(entry.second);
}
}
if (size() < 2 || mCanonicalized) {
// Trivially or already canonical; do nothing.
return *this;
}
std::sort(begin(), end(),
[](auto& a, auto& b) { return keyLess(a.first.get(), b.first.get()); });
mCanonicalized = true;
return *this;
}
std::unique_ptr<Item> Map::clone() const {
auto res = std::make_unique<Map>();
for (auto& [key, value] : *this) {
res->add(key->clone(), value->clone());
}
res->mCanonicalized = mCanonicalized;
return res;
}
std::unique_ptr<Item> SemanticTag::clone() const {
return std::make_unique<SemanticTag>(mValue, mTaggedItem->clone());
}
uint8_t* SemanticTag::encode(uint8_t* pos, const uint8_t* end) const {
// Can't use the encodeHeader() method that calls type() to get the major type, since that will
// return the tagged Item's type.
pos = ::cppbor::encodeHeader(kMajorType, mValue, pos, end);
if (!pos) return nullptr;
return mTaggedItem->encode(pos, end);
}
void SemanticTag::encode(EncodeCallback encodeCallback) const {
// Can't use the encodeHeader() method that calls type() to get the major type, since that will
// return the tagged Item's type.
::cppbor::encodeHeader(kMajorType, mValue, encodeCallback);
mTaggedItem->encode(encodeCallback);
}
size_t SemanticTag::semanticTagCount() const {
size_t levelCount = 1; // Count this level.
const SemanticTag* cur = this;
while (cur->mTaggedItem && (cur = cur->mTaggedItem->asSemanticTag()) != nullptr) ++levelCount;
return levelCount;
}
uint64_t SemanticTag::semanticTag(size_t nesting) const {
// Getting the value of a specific nested tag is a bit tricky, because we start with the outer
// tag and don't know how many are inside. We count the number of nesting levels to find out
// how many there are in total, then to get the one we want we have to walk down levelCount -
// nesting steps.
size_t levelCount = semanticTagCount();
if (nesting >= levelCount) return 0;
levelCount -= nesting;
const SemanticTag* cur = this;
while (--levelCount > 0) cur = cur->mTaggedItem->asSemanticTag();
return cur->mValue;
}
string prettyPrint(const Item* item, size_t maxBStrSize, const vector<string>& mapKeysToNotPrint) {
string out;
prettyPrintInternal(item, out, 0, maxBStrSize, mapKeysToNotPrint);
return out;
}
string prettyPrint(const vector<uint8_t>& encodedCbor, size_t maxBStrSize,
const vector<string>& mapKeysToNotPrint) {
auto [item, _, message] = parse(encodedCbor);
if (item == nullptr) {
return "";
}
return prettyPrint(item.get(), maxBStrSize, mapKeysToNotPrint);
}
} // namespace cppbor
lib/libcppbor/cppbor_parse.cpp
+423
View File
@@ -0,0 +1,423 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include "cppbor_parse.h"
#include <memory>
#include <sstream>
#include <stack>
#include <type_traits>
#include "cppbor.h"
#define CHECK(x) (void)(x)
namespace cppbor {
namespace {
std::string insufficientLengthString(size_t bytesNeeded, size_t bytesAvail,
const std::string& type) {
char buf[1024];
snprintf(buf, sizeof(buf), "Need %zu byte(s) for %s, have %zu.", bytesNeeded, type.c_str(),
bytesAvail);
return std::string(buf);
}
template <typename T, typename = std::enable_if_t<std::is_unsigned_v<T>>>
std::tuple<bool, uint64_t, const uint8_t*> parseLength(const uint8_t* pos, const uint8_t* end,
ParseClient* parseClient) {
if (pos + sizeof(T) > end) {
parseClient->error(pos - 1, insufficientLengthString(sizeof(T), end - pos, "length field"));
return {false, 0, pos};
}
const uint8_t* intEnd = pos + sizeof(T);
T result = 0;
do {
result = static_cast<T>((result << 8) | *pos++);
} while (pos < intEnd);
return {true, result, pos};
}
std::tuple<const uint8_t*, ParseClient*> parseRecursively(const uint8_t* begin, const uint8_t* end,
bool emitViews, ParseClient* parseClient);
std::tuple<const uint8_t*, ParseClient*> handleUint(uint64_t value, const uint8_t* hdrBegin,
const uint8_t* hdrEnd,
ParseClient* parseClient) {
std::unique_ptr<Item> item = std::make_unique<Uint>(value);
return {hdrEnd,
parseClient->item(item, hdrBegin, hdrEnd /* valueBegin */, hdrEnd /* itemEnd */)};
}
std::tuple<const uint8_t*, ParseClient*> handleNint(uint64_t value, const uint8_t* hdrBegin,
const uint8_t* hdrEnd,
ParseClient* parseClient) {
if (value > std::numeric_limits<int64_t>::max()) {
parseClient->error(hdrBegin, "NINT values that don't fit in int64_t are not supported.");
return {hdrBegin, nullptr /* end parsing */};
}
std::unique_ptr<Item> item = std::make_unique<Nint>(-1 - static_cast<int64_t>(value));
return {hdrEnd,
parseClient->item(item, hdrBegin, hdrEnd /* valueBegin */, hdrEnd /* itemEnd */)};
}
std::tuple<const uint8_t*, ParseClient*> handleBool(uint64_t value, const uint8_t* hdrBegin,
const uint8_t* hdrEnd,
ParseClient* parseClient) {
std::unique_ptr<Item> item = std::make_unique<Bool>(value == TRUE);
return {hdrEnd,
parseClient->item(item, hdrBegin, hdrEnd /* valueBegin */, hdrEnd /* itemEnd */)};
}
std::tuple<const uint8_t*, ParseClient*> handleNull(const uint8_t* hdrBegin, const uint8_t* hdrEnd,
ParseClient* parseClient) {
std::unique_ptr<Item> item = std::make_unique<Null>();
return {hdrEnd,
parseClient->item(item, hdrBegin, hdrEnd /* valueBegin */, hdrEnd /* itemEnd */)};
}
template <typename T>
std::tuple<const uint8_t*, ParseClient*> handleString(uint64_t length, const uint8_t* hdrBegin,
const uint8_t* valueBegin, const uint8_t* end,
const std::string& errLabel,
ParseClient* parseClient) {
ssize_t signed_length = static_cast<ssize_t>(length);
if (end - valueBegin < signed_length || signed_length < 0) {
parseClient->error(hdrBegin, insufficientLengthString(length, end - valueBegin, errLabel));
return {hdrBegin, nullptr /* end parsing */};
}
std::unique_ptr<Item> item = std::make_unique<T>(valueBegin, valueBegin + length);
return {valueBegin + length,
parseClient->item(item, hdrBegin, valueBegin, valueBegin + length)};
}
class IncompleteItem {
public:
static IncompleteItem* cast(Item* item);
virtual ~IncompleteItem() {}
virtual void add(std::unique_ptr<Item> item) = 0;
virtual std::unique_ptr<Item> finalize() && = 0;
};
class IncompleteArray : public Array, public IncompleteItem {
public:
explicit IncompleteArray(size_t size) : mSize(size) {}
// We return the "complete" size, rather than the actual size.
size_t size() const override { return mSize; }
void add(std::unique_ptr<Item> item) override {
mEntries.push_back(std::move(item));
}
virtual std::unique_ptr<Item> finalize() && override {
// Use Array explicitly so the compiler picks the correct ctor overload
Array* thisArray = this;
return std::make_unique<Array>(std::move(*thisArray));
}
private:
size_t mSize;
};
class IncompleteMap : public Map, public IncompleteItem {
public:
explicit IncompleteMap(size_t size) : mSize(size) {}
// We return the "complete" size, rather than the actual size.
size_t size() const override { return mSize; }
void add(std::unique_ptr<Item> item) override {
if (mKeyHeldForAdding) {
mEntries.push_back({std::move(mKeyHeldForAdding), std::move(item)});
} else {
mKeyHeldForAdding = std::move(item);
}
}
virtual std::unique_ptr<Item> finalize() && override {
return std::make_unique<Map>(std::move(*this));
}
private:
std::unique_ptr<Item> mKeyHeldForAdding;
size_t mSize;
};
class IncompleteSemanticTag : public SemanticTag, public IncompleteItem {
public:
explicit IncompleteSemanticTag(uint64_t value) : SemanticTag(value) {}
// We return the "complete" size, rather than the actual size.
size_t size() const override { return 1; }
void add(std::unique_ptr<Item> item) override { mTaggedItem = std::move(item); }
virtual std::unique_ptr<Item> finalize() && override {
return std::make_unique<SemanticTag>(std::move(*this));
}
};
IncompleteItem* IncompleteItem::cast(Item* item) {
CHECK(item->isCompound());
// Semantic tag must be check first, because SemanticTag::type returns the wrapped item's type.
if (item->asSemanticTag()) {
#if __has_feature(cxx_rtti)
CHECK(dynamic_cast<IncompleteSemanticTag*>(item));
#endif
return static_cast<IncompleteSemanticTag*>(item);
} else if (item->type() == ARRAY) {
#if __has_feature(cxx_rtti)
CHECK(dynamic_cast<IncompleteArray*>(item));
#endif
return static_cast<IncompleteArray*>(item);
} else if (item->type() == MAP) {
#if __has_feature(cxx_rtti)
CHECK(dynamic_cast<IncompleteMap*>(item));
#endif
return static_cast<IncompleteMap*>(item);
} else {
CHECK(false); // Impossible to get here.
}
return nullptr;
}
std::tuple<const uint8_t*, ParseClient*> handleEntries(size_t entryCount, const uint8_t* hdrBegin,
const uint8_t* pos, const uint8_t* end,
const std::string& typeName,
bool emitViews,
ParseClient* parseClient) {
while (entryCount > 0) {
--entryCount;
if (pos == end) {
parseClient->error(hdrBegin, "Not enough entries for " + typeName + ".");
return {hdrBegin, nullptr /* end parsing */};
}
std::tie(pos, parseClient) = parseRecursively(pos, end, emitViews, parseClient);
if (!parseClient) return {hdrBegin, nullptr};
}
return {pos, parseClient};
}
std::tuple<const uint8_t*, ParseClient*> handleCompound(
std::unique_ptr<Item> item, uint64_t entryCount, const uint8_t* hdrBegin,
const uint8_t* valueBegin, const uint8_t* end, const std::string& typeName,
bool emitViews, ParseClient* parseClient) {
parseClient =
parseClient->item(item, hdrBegin, valueBegin, valueBegin /* don't know the end yet */);
if (!parseClient) return {hdrBegin, nullptr};
const uint8_t* pos;
std::tie(pos, parseClient) =
handleEntries(entryCount, hdrBegin, valueBegin, end, typeName, emitViews, parseClient);
if (!parseClient) return {hdrBegin, nullptr};
return {pos, parseClient->itemEnd(item, hdrBegin, valueBegin, pos)};
}
std::tuple<const uint8_t*, ParseClient*> parseRecursively(const uint8_t* begin, const uint8_t* end,
bool emitViews, ParseClient* parseClient) {
if (begin == end) {
parseClient->error(
begin,
"Input buffer is empty. Begin and end cannot point to the same location.");
return {begin, nullptr};
}
const uint8_t* pos = begin;
MajorType type = static_cast<MajorType>(*pos & 0xE0);
uint8_t tagInt = *pos & 0x1F;
++pos;
bool success = true;
uint64_t addlData;
if (tagInt < ONE_BYTE_LENGTH) {
addlData = tagInt;
} else if (tagInt > EIGHT_BYTE_LENGTH) {
parseClient->error(
begin,
"Reserved additional information value or unsupported indefinite length item.");
return {begin, nullptr};
} else {
switch (tagInt) {
case ONE_BYTE_LENGTH:
std::tie(success, addlData, pos) = parseLength<uint8_t>(pos, end, parseClient);
break;
case TWO_BYTE_LENGTH:
std::tie(success, addlData, pos) = parseLength<uint16_t>(pos, end, parseClient);
break;
case FOUR_BYTE_LENGTH:
std::tie(success, addlData, pos) = parseLength<uint32_t>(pos, end, parseClient);
break;
case EIGHT_BYTE_LENGTH:
std::tie(success, addlData, pos) = parseLength<uint64_t>(pos, end, parseClient);
break;
default:
CHECK(false); // It's impossible to get here
break;
}
}
if (!success) return {begin, nullptr};
switch (type) {
case UINT:
return handleUint(addlData, begin, pos, parseClient);
case NINT:
return handleNint(addlData, begin, pos, parseClient);
case BSTR:
if (emitViews) {
return handleString<ViewBstr>(addlData, begin, pos, end, "byte string", parseClient);
} else {
return handleString<Bstr>(addlData, begin, pos, end, "byte string", parseClient);
}
case TSTR:
if (emitViews) {
return handleString<ViewTstr>(addlData, begin, pos, end, "text string", parseClient);
} else {
return handleString<Tstr>(addlData, begin, pos, end, "text string", parseClient);
}
case ARRAY:
return handleCompound(std::make_unique<IncompleteArray>(addlData), addlData, begin, pos,
end, "array", emitViews, parseClient);
case MAP:
return handleCompound(std::make_unique<IncompleteMap>(addlData), addlData * 2, begin,
pos, end, "map", emitViews, parseClient);
case SEMANTIC:
return handleCompound(std::make_unique<IncompleteSemanticTag>(addlData), 1, begin, pos,
end, "semantic", emitViews, parseClient);
case SIMPLE:
switch (addlData) {
case TRUE:
case FALSE:
return handleBool(addlData, begin, pos, parseClient);
case NULL_V:
return handleNull(begin, pos, parseClient);
default:
parseClient->error(begin, "Unsupported floating-point or simple value.");
return {begin, nullptr};
}
}
CHECK(false); // Impossible to get here.
return {};
}
class FullParseClient : public ParseClient {
public:
virtual ParseClient* item(std::unique_ptr<Item>& item, const uint8_t*, const uint8_t*,
const uint8_t* end) override {
if (mParentStack.empty() && !item->isCompound()) {
// This is the first and only item.
mTheItem = std::move(item);
mPosition = end;
return nullptr; // We're done.
}
if (item->isCompound()) {
// Starting a new compound data item, i.e. a new parent. Save it on the parent stack.
// It's safe to save a raw pointer because the unique_ptr is guaranteed to stay in
// existence until the corresponding itemEnd() call.
mParentStack.push(item.get());
return this;
} else {
appendToLastParent(std::move(item));
return this;
}
}
virtual ParseClient* itemEnd(std::unique_ptr<Item>& item, const uint8_t*, const uint8_t*,
const uint8_t* end) override {
CHECK(item->isCompound() && item.get() == mParentStack.top());
mParentStack.pop();
IncompleteItem* incompleteItem = IncompleteItem::cast(item.get());
std::unique_ptr<Item> finalizedItem = std::move(*incompleteItem).finalize();
if (mParentStack.empty()) {
mTheItem = std::move(finalizedItem);
mPosition = end;
return nullptr; // We're done
} else {
appendToLastParent(std::move(finalizedItem));
return this;
}
}
virtual void error(const uint8_t* position, const std::string& errorMessage) override {
mPosition = position;
mErrorMessage = errorMessage;
}
std::tuple<std::unique_ptr<Item> /* result */, const uint8_t* /* newPos */,
std::string /* errMsg */>
parseResult() {
std::unique_ptr<Item> p = std::move(mTheItem);
return {std::move(p), mPosition, std::move(mErrorMessage)};
}
private:
void appendToLastParent(std::unique_ptr<Item> item) {
auto parent = mParentStack.top();
IncompleteItem::cast(parent)->add(std::move(item));
}
std::unique_ptr<Item> mTheItem;
std::stack<Item*> mParentStack;
const uint8_t* mPosition = nullptr;
std::string mErrorMessage;
};
} // anonymous namespace
void parse(const uint8_t* begin, const uint8_t* end, ParseClient* parseClient) {
parseRecursively(begin, end, false, parseClient);
}
std::tuple<std::unique_ptr<Item> /* result */, const uint8_t* /* newPos */,
std::string /* errMsg */>
parse(const uint8_t* begin, const uint8_t* end) {
FullParseClient parseClient;
parse(begin, end, &parseClient);
return parseClient.parseResult();
}
void parseWithViews(const uint8_t* begin, const uint8_t* end, ParseClient* parseClient) {
parseRecursively(begin, end, true, parseClient);
}
std::tuple<std::unique_ptr<Item> /* result */, const uint8_t* /* newPos */,
std::string /* errMsg */>
parseWithViews(const uint8_t* begin, const uint8_t* end) {
FullParseClient parseClient;
parseWithViews(begin, end, &parseClient);
return parseClient.parseResult();
}
} // namespace cppbor
lib/libcppbor/include/cppbor/cppbor.h
+1141
View File
File diff suppressed because it is too large Load Diff
lib/libcppbor/include/cppbor/cppbor_parse.h
+195
View File
@@ -0,0 +1,195 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#pragma once
#include "cppbor.h"
namespace cppbor {
using ParseResult = std::tuple<std::unique_ptr<Item> /* result */, const uint8_t* /* newPos */,
std::string /* errMsg */>;
/**
* Parse the first CBOR data item (possibly compound) from the range [begin, end).
*
* Returns a tuple of Item pointer, buffer pointer and error message. If parsing is successful, the
* Item pointer is non-null, the buffer pointer points to the first byte after the
* successfully-parsed item and the error message string is empty. If parsing fails, the Item
* pointer is null, the buffer pointer points to the first byte that was unparseable (the first byte
* of a data item header that is malformed in some way, e.g. an invalid value, or a length that is
* too large for the remaining buffer, etc.) and the string contains an error message describing the
* problem encountered.
*/
ParseResult parse(const uint8_t* begin, const uint8_t* end);
/**
* Parse the first CBOR data item (possibly compound) from the range [begin, end).
*
* Returns a tuple of Item pointer, buffer pointer and error message. If parsing is successful, the
* Item pointer is non-null, the buffer pointer points to the first byte after the
* successfully-parsed item and the error message string is empty. If parsing fails, the Item
* pointer is null, the buffer pointer points to the first byte that was unparseable (the first byte
* of a data item header that is malformed in some way, e.g. an invalid value, or a length that is
* too large for the remaining buffer, etc.) and the string contains an error message describing the
* problem encountered.
*
* The returned CBOR data item will contain View* items backed by
* std::string_view types over the input range.
* WARNING! If the input range changes underneath, the corresponding views will
* carry the same change.
*/
ParseResult parseWithViews(const uint8_t* begin, const uint8_t* end);
/**
* Parse the first CBOR data item (possibly compound) from the byte vector.
*
* Returns a tuple of Item pointer, buffer pointer and error message. If parsing is successful, the
* Item pointer is non-null, the buffer pointer points to the first byte after the
* successfully-parsed item and the error message string is empty. If parsing fails, the Item
* pointer is null, the buffer pointer points to the first byte that was unparseable (the first byte
* of a data item header that is malformed in some way, e.g. an invalid value, or a length that is
* too large for the remaining buffer, etc.) and the string contains an error message describing the
* problem encountered.
*/
inline ParseResult parse(const std::vector<uint8_t>& encoding) {
return parse(encoding.data(), encoding.data() + encoding.size());
}
/**
* Parse the first CBOR data item (possibly compound) from the range [begin, begin + size).
*
* Returns a tuple of Item pointer, buffer pointer and error message. If parsing is successful, the
* Item pointer is non-null, the buffer pointer points to the first byte after the
* successfully-parsed item and the error message string is empty. If parsing fails, the Item
* pointer is null, the buffer pointer points to the first byte that was unparseable (the first byte
* of a data item header that is malformed in some way, e.g. an invalid value, or a length that is
* too large for the remaining buffer, etc.) and the string contains an error message describing the
* problem encountered.
*/
inline ParseResult parse(const uint8_t* begin, size_t size) {
return parse(begin, begin + size);
}
/**
* Parse the first CBOR data item (possibly compound) from the range [begin, begin + size).
*
* Returns a tuple of Item pointer, buffer pointer and error message. If parsing is successful, the
* Item pointer is non-null, the buffer pointer points to the first byte after the
* successfully-parsed item and the error message string is empty. If parsing fails, the Item
* pointer is null, the buffer pointer points to the first byte that was unparseable (the first byte
* of a data item header that is malformed in some way, e.g. an invalid value, or a length that is
* too large for the remaining buffer, etc.) and the string contains an error message describing the
* problem encountered.
*
* The returned CBOR data item will contain View* items backed by
* std::string_view types over the input range.
* WARNING! If the input range changes underneath, the corresponding views will
* carry the same change.
*/
inline ParseResult parseWithViews(const uint8_t* begin, size_t size) {
return parseWithViews(begin, begin + size);
}
/**
* Parse the first CBOR data item (possibly compound) from the value contained in a Bstr.
*
* Returns a tuple of Item pointer, buffer pointer and error message. If parsing is successful, the
* Item pointer is non-null, the buffer pointer points to the first byte after the
* successfully-parsed item and the error message string is empty. If parsing fails, the Item
* pointer is null, the buffer pointer points to the first byte that was unparseable (the first byte
* of a data item header that is malformed in some way, e.g. an invalid value, or a length that is
* too large for the remaining buffer, etc.) and the string contains an error message describing the
* problem encountered.
*/
inline ParseResult parse(const Bstr* bstr) {
if (!bstr)
return ParseResult(nullptr, nullptr, "Null Bstr pointer");
return parse(bstr->value());
}
class ParseClient;
/**
* Parse the CBOR data in the range [begin, end) in streaming fashion, calling methods on the
* provided ParseClient when elements are found.
*/
void parse(const uint8_t* begin, const uint8_t* end, ParseClient* parseClient);
/**
* Parse the CBOR data in the range [begin, end) in streaming fashion, calling methods on the
* provided ParseClient when elements are found. Uses the View* item types
* instead of the copying ones.
*/
void parseWithViews(const uint8_t* begin, const uint8_t* end, ParseClient* parseClient);
/**
* Parse the CBOR data in the vector in streaming fashion, calling methods on the
* provided ParseClient when elements are found.
*/
inline void parse(const std::vector<uint8_t>& encoding, ParseClient* parseClient) {
return parse(encoding.data(), encoding.data() + encoding.size(), parseClient);
}
/**
* A pure interface that callers of the streaming parse functions must implement.
*/
class ParseClient {
public:
virtual ~ParseClient() {}
/**
* Called when an item is found. The Item pointer points to the found item; use type() and
* the appropriate as*() method to examine the value. hdrBegin points to the first byte of the
* header, valueBegin points to the first byte of the value and end points one past the end of
* the item. In the case of header-only items, such as integers, and compound items (ARRAY,
* MAP or SEMANTIC) whose end has not yet been found, valueBegin and end are equal and point to
* the byte past the header.
*
* Note that for compound types (ARRAY, MAP, and SEMANTIC), the Item will have no content. For
* Map and Array items, the size() method will return a correct value, but the index operators
* are unsafe, and the object cannot be safely compared with another Array/Map.
*
* The method returns a ParseClient*. In most cases "return this;" will be the right answer,
* but a different ParseClient may be returned, which the parser will begin using. If the method
* returns nullptr, parsing will be aborted immediately.
*/
virtual ParseClient* item(std::unique_ptr<Item>& item, const uint8_t* hdrBegin,
const uint8_t* valueBegin, const uint8_t* end) = 0;
/**
* Called when the end of a compound item (MAP or ARRAY) is found. The item argument will be
* the same one passed to the item() call -- and may be empty if item() moved its value out.
* hdrBegin, valueBegin and end point to the beginning of the item header, the beginning of the
* first contained value, and one past the end of the last contained value, respectively.
*
* Note that the Item will have no content.
*
* As with item(), itemEnd() can change the ParseClient by returning a different one, or end the
* parsing by returning nullptr;
*/
virtual ParseClient* itemEnd(std::unique_ptr<Item>& item, const uint8_t* hdrBegin,
const uint8_t* valueBegin, const uint8_t* end) = 0;
/**
* Called when parsing encounters an error. position is set to the first unparsed byte (one
* past the last successfully-parsed byte) and errorMessage contains an message explaining what
* sort of error occurred.
*/
virtual void error(const uint8_t* position, const std::string& errorMessage) = 0;
};
} // namespace cppbor