-1

What is the cause of this error and how can I get around it? I have reduced the size of the static data as a workaround but is there any way to create static data in haskell that does not run into this error? Basically I want to create large lists of objects that are compiled in and not loaded from file.

Reported here https://ghc.haskell.org/trac/ghc/ticket/14334

Compiling this project file here that is basically a large data object emitted from the rdf4h lib. I wanted to put the data into static space to improve load times and remove IO.

​https://github.com/h4ck3rm1k3/gcc-haskell/blob/master/src/LibData.hs

stack ghci 

gcc-haskell Using main module: 1. Package `gcc-haskell' component exe:gcc-haskell-exe with main-is file: /home/mdupont/experiments/g\ cc-haskell/gcc-haskell/app/Main.hs GHCi, version 8.0.2: http://www.haskell.org/ghc/ :? for help [1 of 3] Compiling LibData ( LibData.hs, interpreted ) ghc: panic! (the 'impossible' happened)

(GHC version 8.0.2 for x86_64-unknown-linux):

getLabelBc: Ran out of labels

Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
h4ck3rm1k3
  • 2,060
  • 22
  • 33
  • what is your question? – epsilonhalbe Oct 08 '17 at 14:50
  • I have edited to include a better question. – h4ck3rm1k3 Oct 08 '17 at 15:09
  • What is the argument you have against - saving the structure in a file and using - read or any other simple parser? IO is not evil - IO is just someone you should have an eye on because he is behaving unexpectedly sometimes. – epsilonhalbe Oct 08 '17 at 16:15
  • I read this from the rdf file using rdf4h. It is slow (1k lines per second). This data is compiler data and I want to feed it eventually as haskell source code to the compiler and have it typed checked. I want to express the types as haskell constructures and load the data into those structures. – h4ck3rm1k3 Oct 08 '17 at 17:23

2 Answers2

1

A quick and dirty solution

What I meant with my comment is something like the following:

Use the set of parsed data structures you have and simplify them a bit (e.g. with regexes)

Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4707" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#op_0" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4777"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4707" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#type" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4776"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4707" "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#addr_expr"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4707" "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" "http://www.w3.org/2002/07/owl#NamedIndividual"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4706" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#op_0" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#5578"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4706" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#type" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#40"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4706" "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#nop_expr"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4706" "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" "http://www.w3.org/2002/07/owl#NamedIndividual"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4705" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#int" LNode plainl2 "619"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4705" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#type" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#93"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4705" "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#integer_cst"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4705" "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" "http://www.w3.org/2002/07/owl#NamedIndividual"
Triple "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#4704" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#op_0" "https://h4ck3rm1k3.github.io/gogccintro/gcc/ontology/2017/05/20/gcc_compiler.owl#5577"

Then write a small parser for your testdata (i know this is incomplete and you might have to tweak this code snippet)

{-# LANGUAGE OverloadedStrings #-}
module Main  where

import Control.Monad (void)
import qualified Data.Text as T
import Data.Text (Text)
import Data.Attoparsec.Text as AT
import qualified Data.Text.IO as TIO

data Triple = Triple Text Text Text

triple :: Parser Triple
triple = do void $ string "Triple"
            skipSpace
            let entry = char '"' *> AT.takeWhile (/= '"') <* char '"'
            a <- entry
            skipSpace
            b <- entry
            skipSpace
            c <- entry
            return $ Triple a b c


main :: IO ()
main = do triples <- map (parseOnly triple) . T.lines <$> TIO.readFile "Libdata.hs"
          print $ length [x | Right x <- triples]

this script runs with stack runhaskell Main.hs in a bit more than a second - fast enough to use it in say automated testing at each save.

A thorough solution

When reading your comments - I think this is somewhat of an XY problem - you want to put data inside your application (I guess for test purposes) because rdf4h is too slow. Taking a glance at the library - it uses parsec - whereas attoparsec would be faster. Saying this I see that you have already filed an issue at github.

What you could also do is clone the git-repo - add a automatic read instance and read the data structure you have successfully parsed and shown.

epsilonhalbe
  • 15,637
  • 5
  • 46
  • 74
  • What i did is just that , read it in, ran show and then exported that to a data load file. That was then provoking this bug. – h4ck3rm1k3 Oct 08 '17 at 19:04
  • no you put the data into the file and compiled it - what I meant add a read instance and read the data from a separate file – epsilonhalbe Oct 08 '17 at 19:23
  • What would be the difference between reading the data like you suggest and using the rdf4h lib to do so, it will still be tainted io. I am looking for basically a DSL to then use like template haskell to later evaluate the c -code contained in there in haskell or translate the data into a new haskell program and run it. – h4ck3rm1k3 Oct 08 '17 at 21:30
  • I thought the issue is that rdf4h is so slow - I do not understand why io is such a problem. – epsilonhalbe Oct 08 '17 at 21:57
  • Ok, thank you I will try out your suggestion and get back to you. I don't know why IO is a problem, I am still learning. – h4ck3rm1k3 Oct 09 '17 at 12:02
0

According to the development team (see bug report https://ghc.haskell.org/trac/ghc/ticket/14334#comment:8) the issue is due to 16 bit label addresses on the module level, the workaround is to split up the code into multiple modules.

h4ck3rm1k3
  • 2,060
  • 22
  • 33