3

I am using nix with reflex platform to compile haskell to javascript with GHCJS 8.0.1. I have written some functions that rely on an internal module in the text library: Data.Text.Internal. The only thing I need from this is the data constructor for Text:

data Text = Text
  {-# UNPACK #-} !A.Array -- payload (Word16 elements)
  {-# UNPACK #-} !Int     -- offset (units of Word16, not Char)
  {-# UNPACK #-} !Int     -- length (units of Word16, not Char)

However, when I try to compile code that uses this data constructor with GHCJS, it fails. The error message makes it clear that somehow, somewhere, the data constructor for Text has been replaced by:

newtype Text = Text JSString

I understand that I can put #ifdefs around blocks of code to work around this. However, I cannot figure out how this data constructor replacement has been accomplished. This is not mentioned anywhere in ghcjs-shims. It isn't mentioned is ghcjs either. Even weirder is that ghcjs-base has functions that use the normal data constructor for Text. I would appreciate if anyone could point me to the source of this replacement.

EDIT: This is not relevant to the question, but the reason I need to use the internals of Text is to efficiently encode things like IPv4 addresses and MAC addresses. In my the benchmarks I included with a previous question, I found that the fastest IPv4 encoding using the public API of text took around 450ns. Using the internals (and writing to a mutable array directly), I can do this in around 35ns, which is at least at 10x improvement in performance.

EDIT 2: Something else I just remembered (which is possibly relevant to the question) is that in another place in my code, I had used hexadecimal from Data.Text.Lazy.Builder.Int. However, with ghcjs, this function isn't exported by this module. I would be glad to contribute it upstream somewhere, but since I don't understand how all of the modules in text are getting shimmed out, I don't know where the appropriate upstream for this is. I would appreciate if someone could point me to the specific file where this is accomplished.

Community
  • 1
  • 1
Andrew Thaddeus Martin
  • 3,225
  • 17
  • 37
  • 2
    Can you go into details on why you need access to the internals of the `Text` representation? – Cactus Dec 02 '16 at 07:12
  • @Cactus I have added details about why I need to access the internals of `Text`. – Andrew Thaddeus Martin Dec 02 '16 at 13:12
  • 1
    The reason I'm asking, is because `Data.Text.Internal` should expose enough functionality to do what you want without looking at the representation itself. Your linked previous question doesn't seem to contain the `Text`-internals-using version so I can't tell for sure yet. – Cactus Dec 02 '16 at 13:18
  • I've looked through the other functions in `Data.Text.Internal`, and it looks like the `text` function does what I need in most situations. However, there are some other text builder abstractions I've been working on where I need something like this: `textToMArray :: Text -> ST s (TA.MArray s)`. This copies a piece of text into a new buffer. I can write this when I have access to the data constructor, but I don't think that anything else in `Data.Text.Internal` lets me do this. I may be able to work around this though. Also, I'd still like to know where the data constructor switcharoo happens. – Andrew Thaddeus Martin Dec 02 '16 at 13:33
  • It would also be nice to see where the existing discussions of this are. I'm guessing that a longer-term goal would be to push the ghcjs-specific stuff into `text` itself (behind `ifdef`s). My cursory perusal of the github issue tracker turns up no talk of this, but I suspect that I just looking for information in the wrong place. – Andrew Thaddeus Martin Dec 02 '16 at 13:38
  • `Data.Text.Array`? – Cactus Dec 02 '16 at 13:43
  • I'm assuming that's in reponse to my comment about textToMArray. Clearly, copyI is half of the solution, but to call this, I have to first get the Array out of the Text. I don't see anything other than the data constructor that lets me do this. – Andrew Thaddeus Martin Dec 02 '16 at 14:51
  • The `JSString` based `text` implementation is available in `Data.JSString` of the `ghcjs-base` package. If it's not available there, that's probably the first place where it should be added. I think it'll take another step to actually get it into the unofficial `text` package replacement offered by reflex-platform after that. – luite Dec 05 '16 at 19:44
  • @luite Thanks. That gives me a better idea of the process for contributing in this area. – Andrew Thaddeus Martin Dec 05 '16 at 20:18

1 Answers1

6

It looks like you might be using reflex-platform. We recently integrated a change that uses JSString for Text rather than the usual implementation; that's because JSString is dramatically faster and uses a lot less memory than the pure-Haskell Text implementation.

If this is what you're running into, I'd encourage you to stick with JSString unless you're quite unconcerned about performance. However, if you'd like to disable this optimization, I think it should work to just comment out this block and then re-enter your try-reflex/work-on/nix-shell.

Ryan Trinkle
  • 111
  • 3
  • Thanks! I'm definitely interesting in leaving this optimization turned on, and since the `text` smart constructor is still available, I can still work on mutable arrays and freeze them when needed. If I have other general feedback about this feature, where is the most appropriate place to leave it? On the github issue page for reflex-platform or somewhere else? – Andrew Thaddeus Martin Dec 05 '16 at 19:32
  • A github issue on reflex-platform works well, or for live discussion you can join us in #reflex-frp on freenode.irc.net. – Ryan Trinkle Dec 10 '16 at 20:52