How to pack / unpack a hex string (high nibble first) with Elixir

Question

I was wondering how I would work with hex strings in Elixir. Specifically, I'm interested in converting from Hex to ASCII.

In Ruby, an implementation of this may be:

["001C7F616A8B002128C1A33E8100"].pack('H*').gsub(/[^[:print:]]/, '.')

How would I accomplish this task with Elixir? I have tried:

<<00, 01, C7, F6...>>

but this isn't a correct representation of the hex for a string. Thanks for your time and assistance!

So I've made some progress but am currently struggling with the recursion aspect of this.

This is my solution thus far:

defmodule ElixirNetworkTools do
  def decode(payload) do
    upper_payload = String.upcase payload
    case Base.decode16(upper_payload) do
      :error -> decode_with_nonprintable_characters(payload)
      {:ok, decoded_payload} -> decoded_payload
    end
    |> IO.write
  end

def decode_with_nonprintable_characters(payload) do
String.chunk(payload, ~r/\w{2}/)
|> Enum.each(fn(byte) ->
  case Base.decode16(byte) do
    :error -> '.'
    {:ok, decoded_payload} -> decoded_payload
      end
    end)
  end
end

José Valim · Accepted Answer · 2015-05-10T06:58:49.407

Here is another solution to the problem. A couple things before we start:

You can pass case: :mixed to Base.decode16/2: Base.decode16(string, case: :mixed), for this reason, you don't need do upcase before.
If you are going to raise on an invalid string, don't bother checking, just call decode16 directly as it also checks the size.

This means we can start with:

decoded = Base.decode16!(string, case: :mixed)

Now you need to replace non-printable characters. Don't use String.printable?/1 because it is about UTF-8 and not ASCII. We need to implement our own function but what makes more sense: to raise or replace them? It seems it must be considered an error if someone send invalidate data? If that is the case:

def validate_ascii!(<<h, t::binary>>) when h <= 127 do
  validate_ascii!(t)
end

def validate_ascii!(<<>>) do
  true
end

def validate_ascii!(rest) do
  raise "invalid ascii on string starting at: #{rest}"
end

Alternatively you can just remove the last clause and it fail too.

Now we can put it together:

decoded = Base.decode16!(string, case: :mixed)
validate_ascii!(decoded)
decoded

EDIT: If you need to replace non-ascii by dots:

def keep_ascii(<<h, t::binary>>, acc) when h <= 127 do
  keep_ascii(t, acc <> <<h>>)
end

def keep_ascii(<<_, t::binary>>, acc) do
  keep_ascii(t, acc <> ".")
end

def keep_ascii(<<>>, acc) do
  acc
end

Thanks for this solution! It was really helpful to see your way of approaching the problem. Sadly, while I couldn't use it as the desire was to replace as I am working to build a hexadecimal payload inspector, so it is possible / likely that I'll need to display the characters as a period. With that said, this was really helpful to see another way of handling this without explicit checking but via pattern matching solutions. Thanks! Now to do your tutorial at https://howistart.org/posts/elixir/1 — kkirsche, May 10 '15 at 02:53
Glad to help! I have added an example on how to replace non-ascii by dots. — José Valim, May 10 '15 at 06:59

score 0 · Answer 2 · answered May 07 '15 at 20:49

The solution ended up being as follows, though if there is a cleaner or better solution I would be very interested in knowing it.

defmodule ElixirNetworkTools do
  @doc """
  The decode function takes a hexadecimal payload, such as one generated
  by Snort, and returns the ASCII representation of the string.

  ## Example
  iex> ElixirNetworkTools.decode("436F6E74656E742D4C656E6774683A203132")
  {:ok, "Content-Length: 12"}
  """
  def decode(payload) do
    case _validate_length_of_snort(payload) do
      :error -> raise "Invalid length hex string. Must be even length. Exiting"
      _ -> nil
    end

    decoded = String.upcase(payload)
    |> _do_decode
    |> to_string

    {:ok, decoded}
  end

  @doc """
  Internal function used to manually process the hexadecimal payload,
  and builds a char list of the printable characters. If a character is
  not printable, we instead use periods.

  ## Example
  iex> ElixirNetworkTools._do_decode("436F6E74656E742D4C656E6774683A203132")
  ["Content-Length: 12"]
  """
  def _do_decode(payload) do
    Base.decode16!(payload)
    |> String.chunk(:printable)
    |> Enum.map(fn(chunk) ->
        case String.printable? chunk do
          true -> chunk
          _ -> "."
        end
    end)
  end

  @doc """
  Internal function used to validate the length of the hexadecimal payload.
  Hexadecimal strings should have an even number of characters.

  ## Example
  iex> ElixirNetworkTools._validate_length_of_snort("436F6E74656E742D4C656E6774683A203132")
  :ok
  """
  def _validate_length_of_snort(payload) do
    String.length(payload)
    |> rem(2)
    |> case do
      0 -> :ok
      _ -> :error
    end
  end
end

How to pack / unpack a hex string (high nibble first) with Elixir

2 Answers2