no_std support

[wildarch]

Resolves #79.

This builds on the efforts of @chrysn to implement no_std support. It also introduces a few things I needed when developing an embedded application based on this crate.

Notable changes

• Adds a default std feature that can be disabled for no_std support
• Introduction of a local Write trait similar to the existing Read, as std::io::Write is not available in core.
• SliceReadFixed, similar to SliceRead but using a mutable slice as a scratch buffer. This is useful in no_std environments when the input slice is immutable (so MutSliceRead is off the table).
• A new feature unsealed_read_write (disabled by default) becomes available, which unseals the Read and Write trait. This is useful in no_std to implement Read on a custom File type.

Public API Changes

the methods Serializer::new, Serializer::packed and Serializer::new_with_options now take a writer that implements the local Write trait instead of std::io::Write. An std::io::Write now needs to be wrapped in an IoWrite to be passed to any of these functions. In most cases this should not be a problem, as the types for serde_cbor::to_vec and serde_cbor::to_writer still take std::io::Write.

One possible alternative would be to make new, packed, new_with_options still take std::io::Write and add similar methods for Write.

Papercuts

• In no_std mode, deserialiation and serialiation errors are mapped to a generic error that removes all context.
• The Read and Write traits are duplicated for the unsealed_read_write feature, this is not very DRY.

[wildarch]

Whoops, it seems like the ser module was accidentally feature gated on std, meaning serialization does not currently work without std. I'll resolve this and mark the PR as WIP for now.

[wildarch]

The Vecs necessary for canonicalization of structs and maps make the current serialization impossible to use without alloc. Maybe canonicalization could be made a setting that is forced to off in no_std environments?

[pyfisch]

The Vecs necessary for canonicalization of structs and maps make the current serialization impossible to use without alloc. Maybe canonicalization could be made a setting that is forced to off in no_std environments?

I am currently rewriting the value module. I think about always serialize values in canonical form. So this can be resolved later.

[pyfisch]

great!

the methods Serializer::new, Serializer::packed and Serializer::new_with_options now take a writer that implements the local Write trait instead of std::io::Write. An std::io::Write now needs to be wrapped in an IoWrite to be passed to any of these functions. In most cases this should not be a problem, as the types for serde_cbor::to_vec and serde_cbor::to_writer still take std::io::Write.

Is it possible that they take Into<Write> and io::Write is convert by the called function?

[pyfisch]

What should be done here?

[wildarch]

Right now we just set the code to ErrorCode::Custom without any extra data on what caused the error. Ideally I would like to propagate the error message here, but that would only be possible if we could Box up the message. The next best thing would be to propagate part of the error, but I have no good ideas for how to do that at the moment.

[pyfisch]

see above

[pyfisch]

Btw can you please test no-std on Travis CI?

[wildarch]

great!

the methods Serializer::new, Serializer::packed and Serializer::new_with_options now take a writer that implements the local Write trait instead of std::io::Write. An std::io::Write now needs to be wrapped in an IoWrite to be passed to any of these functions. In most cases this should not be a problem, as the types for serde_cbor::to_vec and serde_cbor::to_writer still take std::io::Write.

Is it possible that they take Into<Write> and io::Write is convert by the called function?

I have tried to get it to work with Into<Write>, but ran into issues with lifetimes, as we also need to be able to pass in a &mut Write (which implements Write itself). Do you have any suggestions how we might work around this?

[wildarch]

Btw can you please test no-std on Travis CI?

Sure, I will add some tests for no-std mode.

[pyfisch]

I have tried to get it to work with Into, but ran into issues with lifetimes, as we also need to be able to pass in a &mut Write (which implements Write itself). Do you have any suggestions how we might work around this?

Didn't think about this. We can merge the PR as it is now.

[wildarch]

Great! I actually have a fix for serialization in a local branch we might also want to incorporate before merging? I should have time on Tuesday to push those changes and implement a few tests for no_std

[pyfisch]

Ok. wildarch <[email protected]> schrieb am So., 10. März 2019, 21:09:

…

Great! I actually have a fix for serialization in a local branch we might also want to incorporate before merging? I should have time on Tuesday to push those changes and implement a few tests for no_std — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#96 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACpvWdh72vWjMSdItcpqiSFWpWelkVsqks5vVWZpgaJpZM4bT96I> .

[wildarch]

Serialization and is now working on no_std, with CI tests 🎉

I think this should be fine to merge now 👍

[chrysn]

Thanks a bunch for finishing this.

I just tried to port my application over, which means that windowed_infinity will implement the unsealed Write trait -- but so far I failed with the next-release version as the Write trait only lives in the private write module. Did I miss a mechanism here, or is Write still actually sealed?

If I did not miss any way of implementing Write as it is, that'd mean that the original way of sealing Read was superfluous (as users already can't implement the trait if you cant' name it), and the unsealing might become as easy as having a conditional pub mod unsealed { pub use write::Write, read::Read; } in the lib.

[chrysn]

My bad: the Write trait is exposed as serde_cbor::ser::Write and thus public, and also usable for windowed_inifinity (which now implements the trait correctly).

[wildarch]

@chrysn happy to help 😄. The Write trait is indeed publicly exposed even if unsealed_read_write is disabled. I am wondering if your suggestion might work if we were to not expose the Read and Write traits by default? I think users of the crate only need it when they create a Serializer, as the new function is generic over that trait. I guess since we have methods like serde_cbor::to_vec, we could get away with not exposing the trait?

[chrysn]

If there were no reason to expose them then it might work, but seeing how it is exposed now (and can be used) leads me to think that it does make sense to implement the trait, if only to allow others to have their interfaces generic over it – so sealing probably needs to stay in some form (though I have #98 as a proposal for how it can stay in a way that works better with versioning).