CGlue Update - Greatness Is Near!

Published: May 27, 2021, 11:51 p.m.

A few days ago I introduced a seamless ABI safety solution, and there has been rapid progress on it. It's very close to being feature complete, but there are some final important kinks that need working out.

Revisit CGlue

In short, CGlue aims to solve the pains of writing functional dynamically loadable Rust code. The aim of the project is to generate wrapper structures and implementations for traits and their groups, which can then be passed over to FFI-side, without changing a single line of the trait itself. In addition, cbindgen should be able to generate C headers for these structures. Basically - stable, C-compatible trait objects with some extra features sprinkled on top.

Current progress

An example I gave in my last post involved a single annotated trait, which could then be cast into a CGlue object with one macro invocation:

let obj = trait_obj!(info as InfoPrinter); // previously cglue_obj!

I also mentioned possibility of grouping traits together, as well as defining optional traits. I'm happy to say that this entire endeavor has been successful! Let's break it all down.

First, we define some types and traits

struct SA {}
struct SB {}

#[cglue_trait]
pub trait TA {
    extern "C" fn ta_1(&self) -> usize;
}

impl TA for SA { ... }
impl TA for SB { ... }

#[cglue_trait]
pub trait TB {
    extern "C" fn tb_1(&self, val: usize) -> usize;
    fn tb_2(&self, val: usize) -> usize;
}

impl TB for SB { ... }

#[cglue_trait]
pub trait TC {
    fn tc_1(&self);
    extern "C" fn tc_2(&mut self);
}

impl TC for SA { ... }

TA is shared on both types, whereas TB, and TC are implemented on a single type.

Then, we declare a trait group

cglue_trait_group!(TestGroup, TA, { TB, TC });

Here, a TestGroup is defined, that requires TA trait to be implemented everywhere, but TB and TC are optional.

Define types that implement the group

cglue_impl_group!(SA, TestGroup, { TC });
cglue_impl_group!(SB, TestGroup, { TB });

SA implements optional TC, while SB implements optional TB. This step could have been avoided if we had specialization or negative bounds, but alas, life is not perfect sometimes. However, everything is type checked anyways, the worst that could happen is a type not advertising support for a trait, but that would still be safe.

With all this done, CGlue will behind the scenes generate all needed functions for all combinations of trait usage. If curious, have a look :)

Finally, use the code

You can use group_obj macro to transform a &T, &mut T, or T into a group:

let a = SA {};
let _ = group_obj!(&a as TestGroup);
let group = group_obj!(a as TestGroup);

Upon constructing a group, everything is type checked to ensure required vtables can be generated, and then, T is asked to append the vtable list with optional ones. Afterwards, type information is fully destroyed. Reference types become c_void, moved types become CBox<c_void>, and then there is no turning back.

as_ref!, as_mut!, cast!, or into! macros can be used to access optional traits:

{
    let group = as_ref!(group impl TC)?;
    group.tc_1();
}

check! macro allows to see beforehand if the conversion is valid:

assert!(!check!(group impl TB));

You can also just try to get a reference...

assert!(as_mut!(group impl TB).is_none());

Using cast!, as opposed to into!, allows to retrieve the original group back:

let cast = cast!(group impl TC)?;
let mut group = TestGroup::from(cast);

As you can see, the interface is rather powerful already. I don't think you can get optionality working with regular trait objects, without resorting to reflection. And good luck doing that safely.

Now, midway through this process I found abi_stable crate, and it does seem to have a very similar scope. It has been an ongoing project for many years too! Sadly, it does not seem to be active anymore and there are several extra features I wanted to expand upon. Thus, all this work may not be all for nothing!

Future

Being frictionless is the best way to put it. That means not requiring to change function signatures for safety. The crate already generates thin extern "C" wrappers for trait functions that use the Rust-native calling convention. This is the current wrapper generation code:

let name = &self.name;
let args = self.vtbl_args();
let out = &self.out;
let call_args = self.chained_call_args();

let trname = &self.trait_name;
let fnname = format_ident!("{}{}", FN_PREFIX, name);

let gen = quote! {
    extern "C" fn #fnname<T: #trname>(#args) -> #out {
        this.#name(#call_args)
    }
};

However, it is not super versatile, and does not handle unsafe arguments. Thus, there are some code generation additions needed:

  1. Converting slice parameters to pointer and usize pairs.

  2. Optionally wrapping Result<T, E> types to write Ok result to a MaybeUninit<T> parameter, if the error type is convertible to NonZero integer. This would allow passing outputs without extra cost, and C users would be extra happy.

  3. Wrapping other Result<T, E> types into safe CResult enum, as well as nullable Option types as COption.

  4. Support self-consuming function calls.

  5. Wrapping associated types to different CGlue objects. This one's probably the trickiest. It involves 2 parts:

5.1. Basic wrapping when returning associated types. It could be as simple as adding trait_obj!(ret as Trait) when returning.

5.2. Complex wrapping when returning references to associated types. This would require to reserve space on the self object beforehand, where the wrapped object would be stored. With mutable calls it would be as simple as storing MaybeUninit<AssociatedType>, but with immutable calls, interior mutability would need to be used. As you can see, a lot of edge cases...

  1. Any other edge cases that would arise.

I'll be honest, I'm not looking forward to this bit... However, perhaps writing this down already helped me define the problem :)

I think the final piece of the puzzle would have to do with actual plugin loading, and runtime version validation. I haven't really given much thought about it, perhaps something like type hashing would work? I don't know, I guess we'll see as time goes.

But other than that, I think it is not too far away from being integratable to memflow, and usable for other projects as well! This is the point where I ask you if there is anything else missing :D Don't hesitate to shoot me a message (if it's not July yet)! I believe this is a solid feature set as it is, but there could be some key parts missing. I guess time will tell.