0

I'm using HTTPoison to get the elixir guide website and then parsing it with Floki to build a HTML 2 Jupyter Notebook transformer (with Markdown for the description). I have to put in the ` backtick. \u0060 for code highlighting, which works so far. I have some places where I use string Interpolation "#{Floki.text(childs_nodes)}" and in other Places Enum.join "" to process and transform from HTML to Markdown.

The transformed result is stored in a map according to the jupyter notebookformat. When I call Poison.encode notebook I get an error, as the codepoints are gone. I tried different things, but don't know yet, where the issue is.

Any hints what I'm doing wrong when processing the text? This is the exception:

** (Poison.EncodeError) unable to encode value: {:source, ["Elixir also    provides `Port`, `Reference` and `PID` as data types (usually used in process communication), and we will take a quick look at them when talking about processes. For now, let’s take a look at some of the basic operators that go with our basic types."]}
lib/poison/encoder.ex:377: Poison.Encoder.Any.encode/2
lib/poison/encoder.ex:255: anonymous fn/3 in Poison.Encoder.List.encode/3
lib/poison/encoder.ex:256: Poison.Encoder.List."-encode/3-lists^foldr/2-1-"/3
lib/poison/encoder.ex:256: Poison.Encoder.List.encode/3
lib/poison.ex:41: Poison.encode!/2
(guide2nb) lib/cli.ex:27: CLI.process/1
(elixir) lib/kernel/cli.ex:76: anonymous fn/3 in Kernel.CLI.exec_fun/2
Sheharyar
  • 73,588
  • 21
  • 168
  • 215
Rainer Schuster
  • 102
  • 7

1 Answers1

2

The problem here is that you are trying to encode a Tuple, while Poison only works with Maps and Lists. If the value you were trying to encode was a Map instead of a tuple, it would work perfectly. Unicode has nothing to do with this.

iex(1)> value = %{source: ["Elixir also    provides `Port`, `Reference` and `PID` as data types (usually used in process communication), and we will take a quick look at them when talking about processes. For now, let’s take a look at some of the basic operators that go with our basic types."]}
iex(2)> Poison.encode(value)
{:ok,
 "{\"source\":[\"Elixir also    provides `Port`, `Reference` and `PID` as data types (usually used in process communication), and we will take a quick look at them when talking about processes. For now, let’s take a look at some of the basic operators that go with our basic types.\"]}"}
Sheharyar
  • 73,588
  • 21
  • 168
  • 215
  • Thanks for the hint ... Going to refactor my code. Actually I'm only using maps and lists, but I have some flat_maps in the recursion of parsing the HTML tree. So some of my maps `%{cell_type: :code, metadata: %{}, source: ["0o777"]},` get converted to keyword lists with tuples. `[{:cell_type, :markdown}, {:metadata, %{}}, {:source, ["# Basic types"]}, {:cell_type, :markdown}, {:metadata, %{}}, {:source, []},` – Rainer Schuster Nov 03 '16 at 05:28
  • It could be useful to change the name of the question to closer represent what the issue was. It would make it an easier reference for others. – mmartinson Oct 19 '17 at 00:07