Re: ByteArray / Uint8Array

[philip chimento gmail com]

Hi Andrea and Giovanni,

Thanks for the replies and for thinking about the problem.

On Tue, Nov 7, 2017 at 3:39 PM Andrea Giammarchi <andrea giammarchi gmail com> wrote:

just thinking about this, how about you keep ByteArray as it is, deprecating it, and you introduce a global Buffer still based on UInt8Array ?

It's unfortunately not possible to keep everything as it is while still switching to Uint8Array - functions that take a byte array as an input parameter could accept any of ByteArray, Uint8Array or a new Buffer class, but functions that return a byte array would need to choose which one they return. If they continue to return ByteArray then there's not as much benefit in switching, in my opinion.

 

in cgjs I've just done `require("buffer")` which brings in the node version of a buffer which is already based on native Uint8Array

I will check that out, thanks.

 

it'd be awkward to move back and forward from ByteArray <=> Buffer/Uint8Array in the transition time, so having a new primitive already well documented might be a better option?

otherwise I personally wouldn't care as long as it's backward compatible

It may be possible to write a work-alike ByteArray replacement in JS, even if it's not zero-copy. I've started to do that but have not had time to continue it the past few days. Hopefully the Buffer code might give me some ideas.

On Tue, Nov 7, 2017 at 10:42 PM Giovanni Campagna <giocampagna92 gmail com> wrote:

On Tue, Oct 31, 2017 at 10:13 PM, <philip chimento gmail com> wrote:

Hi list,

I have some code in progress to revamp GJS's ByteArray to use a standard _javascript_ Uint8Array (part of modern _javascript_'s typed array family) instead of the custom GJS object. Read more about it here: https://gitlab.gnome.org/GNOME/gjs/issues/5

Does anyone have any thoughts on ByteArray or see any problems with the proposed course of action?

One of the reasons ByteArray exists is that it wraps GByteArray and GBytes from glib.

In particular for GBytes, it means that, as long as you don't modify the underlying store, you can pass around binary data between libraries with no copies.

I don't know if JS UInt8Arrays allow that, and I worry they would not - at the very least, it would be hard to do the copy-on-write behavior ByteArray has.

There is JS_NewArrayBufferWithExternalContents() [1], but it looks like it won't exactly play well with a refcounted GBytes. What would really be nice here is a way to create an ArrayBuffer that calls a callback when it's released so that you can unref the GBytes, but that doesn't exist.

At the same time, as you point out in the bug, GLib.Bytes exists and would not go away.

It could be extended to do field access so that JS can read the actual data, removing the need to wrap it in a zero-copy ByteArray.

That may even already work, I'm not sure. I will check. I like the idea of getting a Uint8Array from GBytes.

[1] https://searchfox.org/mozilla-central/search?q=JS_NewArrayBufferWithExternalContents&case=false&regexp=false&path=