All posts

Migrating to Elixir's Earmark for markdown processing

Prior to migrating to Earmark any html needed for the DevDeck cards was hard coded. Switching to Earmark for markdown processing now saves me a lot of time in card creation. Previously I would have to save a card answer with a code block like this:

"""String of answer text
  <br />
  <pre><code>IO.inspect "some code"</code></pre>
"""

Now I can accomplish the same thing with:

"""String of answer text
  \n
  ```IO.inspect "some code"</code></pre>```
"""

It might not look like a big deal but the minutia of getting the html formatted correctly for each of the of questions and answers is cumbersome. And if you’re unfamiliar with markdown processors, they essentially take a set of rules and convert those rules to html nodes. For example the open and closed backticks above get converted the <pre><code></code></pre> html nodes for displaying code snippets in the browser.

The migration to Earmark included three steps:

  1. Adding Earmark as a project dependency
  2. Using the Earmark client to process markdown
  3. Writing and executing a migration against existing cards to convert hard coded html to markdown.

Step one: adding Earmark as a project dependency

defp deps do
  [
    ...
    {:earmark, ">= 1.4.15"}
  ]
end

And update dependencies with mix deps.get.

Step two: using the Earmark client to process markdown

Then use the Earmark client where the html calls for the cards. For this application I am using LiveView for websocket based client/server data transactions so I added the Earmark functionality to my card_live.ex file which will include the card data in the rendering of the card.html.leex template. I use the as_html function on the Earmark module to accomplish this. That update looked like:

card_live.ex


def mount(%{"uuid" => uuid} = params, _session, socket) do
  cards = Card.from_uuid(uuid)
  cards = Enum.map(cards, fn (card) ->
    {:ok, answer, _opt} = Earmark.as_html(card.answer)
    {:ok, question, __opt} = Earmark.as_html(card.question)
    %{card | answer: answer, question: question}
  end)

  {:ok, assign(socket, cards: cards)}
end

Above I am mapping over a list of cards and processing the question and answer markdown before assigning the cards to the socket.

The rendering didn’t change but this is what the template code looks like:

card.html.leex

<%= @cards |> Enum.with_index |> Enum.map(fn({card, index}) ->  %>
  <div id="card">
    <span><%= raw card.question %></span>
    <span><%= raw card.answer %></span>
  </div>
<% end) %>

Step 3: Writing and executing a migration against existing cards to convert hard coded html to markdown.

First generate a new migration file on the command line through:

mix ecto.gen.migration earmark_cards

And then in the migration file I convert all of the question and answer strings using String.replace and replace the html code matched with its markdown equivalent, I annotate some of the script where I think it could be helpful.

defmodule DevDecks.Repo.Migrations.EarmarkCards do
  use Ecto.Migration

  def up do
    # Get all decks and then all the cards from those decks
    DevDecks.Deck.query_uuids |> Enum.map(fn(uuid) -> DevDecks.Card.from_uuid(uuid) end)
    |> Enum.map(fn(card_set) ->
      Enum.map(card_set, fn(card) ->
        # some test cards don't have answers so add a default
        answer = card.answer || ""
        # pipe the answer string through all of the regex string replacement
        updated_answer = answer
        |> String.replace(~r/<pre><code>/, "```")
        |> String.replace(~r/<\/code><\/pre>/, "```")
        |> String.replace(~r/<code>/, "`")
        |> String.replace(~r/<\/code>/, "`")
        |> String.replace(~r/<br \/>/, "\n")
        |> String.replace(~r/<br\/>/, "\n")
        |> String.replace(~r/<br>/, "\n")

        # repeat steps for question (this could have been extracted to a function)
        question = card.question || ""
        updated_question = question
        |> String.replace(~r/<pre><code>/, "```")
        |> String.replace(~r/<\/code><\/pre>/, "```")
        |> String.replace(~r/<code>/, "`")
        |> String.replace(~r/<\/code>/, "`")
        |> String.replace(~r/<br \/>/, "\n")
        |> String.replace(~r/<br\/>/, "\n")
        |> String.replace(~r/<br>/, "\n")

        # call the update function on the card context with the data uuid to find the card and the updated answer and question.
        DevDecks.Card.update(%{"uuid" => card.uuid, "answer" => updated_answer, "question" => updated_question})
      end)
    end)
  end

  # add a down method for rollbacks if needed.
  def down
    nil
  do
end

After testing this locally I deployed it using Gigalixir and ran the migration in production from the command line through gigalixir run mix ecto.migrate and could watch the migration through the logs through gigalixir logs.

Following the successful migration the transition was complete.