vxml - v1.0.0

VXML

This package is the reference implementation of VXML (“Vanilla XML”), a datatype and document format representing a simplified subset of XML for document processing and markup-language transpilation.

From XML, VXML keeps only recursive nodes, attributes, and text nodes. Other XML features are not expressible in VXML.

VXML is intended to operate as an intermediate between different light-markup-style document formats. A parser can convert a source document into VXML, a pipeline can transform the AST, and an emitter can serialize the result to HTML, XML-like text, JSX, or any other target for which an emitter has been written. VXML’s simple shape encourages equally simple encoding and decoding contracts.

VXML comes with its own indentation-based serialization format for human inspection and for persisting documents required by test suites.

The in-program VXML datatype also conveys blame from the source document or an intervening transformation pipeline. Each atomic unit such as a tag, an attribute, or a line of text carries a Blame. This provides a built-in traceability mechanism for document transpilation. Blames are not encoded in VXML’s default serialization but a specific emitter can choose to be blame-aware, e.g., to provide “click to jump back to source”-type functionality.

Example

This code parses an XML file to VXML and serializes the result as pretty-printed HTML:

import gleam/result
import gleam/string
import simplifile
import vxml
import vxml/blame
import vxml/io_lines

pub fn xml_file_to_html(path: String) -> Result(String, #(blame.Blame, String)) {
  simplifile.read(path)
  |> result.map_error(fn(e) { #(blame.no_blame, string.inspect(e)) })
  |> result.try(vxml.parse_xml(_, path))
  |> result.map(vxml.vxml_to_html_output_lines(_, 0, 2))
  |> result.map(io_lines.output_lines_to_string)
}

Package Contents

This package includes:

Model

The supporting data-bearing types are:

The main type is:

pub type VXML {
  V(blame: Blame, tag: String, attrs: List(Attr), children: List(VXML))
  T(blame: Blame, lines: List(Line))
}

Here:

Blame aside, VXML is built on four data-bearing types: V, T, Line, and Attr.

Moreover, each V, T, Line, and Attr value carries one Blame, stored as its first field, for a simple one-blame-per-value mental model.

VXML is semantics-agnostic: tags and attributes are names, not behaviors.

Serialized Format

VXML includes a compact text format used for round-tripping, tests, and debug output. Its serialization is indentation-based and canonical: each VXML tree has one serialized VXML form.

A caret-like marker opens a node, attributes appear underneath the tag, and quoted lines form text nodes:

<> Article
  id=intro
  <> Title
    <>
      'A dark and stormy night'
  <> Section
    <> SectionTitle
      <>
        'Darkness descends'
    <> Paragraphs
      <>
        'This is the third text node'
        'of the tree, but the first'
        'text node with >1 lines.'
      <>
        'For VXML, this is just a'
        'second text node. A "paragraph"'
        'is not one of VXML's abstractions.'

Each line of text appears as a visible quoted line, making newline placement and whitespace easy to spot by human inspection.

Rules:

Relevant rules also apply to the VXML datatype itself, even while the type system is not able to encode some constraints, e.g., the fact that = is an invalid character inside an attribute key. These constraints could have been enforced via opaque types, but the present package takes a more live and let live approach, which has advantages in terms of allowing transforms to directly case on VXML values, etc. But note that a VXML payload is considered malformed if it contains a T-node with an empty list of lines.

Serialized VXML can be parsed and emitted directly:

let assert Ok([tree]) =
  vxml.parse_string(source, "example.vxml", True)

let text =
  vxml.vxml_to_string(tree)

Ingress: Parsing XML and HTML

The default XML-like parser takes a source string and a filename-like token to use for blame-generation:

let path = "content/source.xml"
let short_pathname_to_use_in_blame = "source.xml"

simplifile.read(path)
|> result.map_error(fn(e) { #(blame.no_blame, string.inspect(e)) })
|> result.try(vxml.parse_xml(_, short_pathname_to_use_in_blame))

For iffy input that may come from a handwritten HTML source, html_repair can repair a few common patterns before parsing:

let path = "content/source.html"
let short_pathname_to_use_in_blame = "source.html"

simplifile.read(path)
|> result.map_error(fn(e) { #(blame.no_blame, string.inspect(e)) })
|> result.map(vxml.html_repair)
|> result.try(vxml.parse_xml(_, short_pathname_to_use_in_blame))

The html_repair step:

The individual repair helpers are public so callers can apply only the repair steps they want. These helpers are deliberately narrow string repairs, not a general HTML parser.

XML comments are tokenized by the lower-level streamer, but parse_xml does not represent them in the returned VXML tree.

Before parsing, source strings are converted to List(InputLine). That conversion can be performed directly with io_lines.string_to_input_lines, and the result can be inspected with io_lines.input_lines_table. For even lower-level inspection one can use xml_streamer.input_lines_streamer, which turns those input lines into XML token events rather than VXML.

HTML and JSX Output

Use the HTML helpers when a VXML tree directly represents HTML elements. The output stays line-based until it is converted to a string or written to disk:

let lines = vxml.vxml_to_html_output_lines(tree, 0, 2)

The HTML serializer escapes non-entity ampersands in text. It treats common inline tags as sticky when laying out output, so inline content is not forced onto separate lines unless the tree requires it.

JSX-like output is available through:

let lines = vxml.vxml_to_jsx_output_lines(tree, 0, 2)
let source = vxml.vxml_to_jsx(tree, 0, 2)

Blame

Every node, attribute, and line carries a Blame value. Blame records where a piece of data came from, or which later transformation introduced it.

pub type Blame {
  Src(comments, path, line_no, char_no, cursor)
  Des(comments, name, line_no) // maintained desugarer code
  Ext(comments, name)          // external/manual code attribution
  NoBlame(comments)
}

SourceCursor controls whether source positions can move when text is sliced:

This is useful for parser and transformation pipelines that need diagnostics or source maps after several tree rewrites.

Des and Ext can be used for code-attributed blame, respectively from inside a transformation pipeline and from outside it, such as an emitter step.

Blame Tables

Use vxml_table to inspect serialized VXML together with its attached blames. For direct control over the emitted lines, use vxml_to_output_lines together with io_lines.output_lines_table_with. This allows the blame margin columns to be sized explicitly:

let assert Ok([tree]) =
  vxml.parse_string(source, "example.vxml", True)

tree
|> vxml.vxml_to_output_lines
|> io_lines.output_lines_table_with(
  "",
  0,
  blame.BlameTableMarginColumnsMinMax(30, 30),
  blame.BlameTableMarginColumnsMinMax(0, 0),
)
|> io.println

The first BlameTableMarginColumnsMinMax controls the blame digest columns. The second controls the blame comments columns. Passing (0, 0) for the comments columns suppresses them entirely. For the serialized VXML example above, this prints:

┌────────────────────────────────────────────────────────────────────
│ Blame                       █doc
├────────────────────────────────────────────────────────────────────
│ example.vxml:1:1 ->         █<> Article
│ example.vxml:2:3            █  id=intro
│ example.vxml:3:3 ->         █  <> Title
│ example.vxml:4:5            █    <>
│ example.vxml:5:7            █      'A dark and stormy night'
│ example.vxml:6:3 ->         █  <> Section
│ example.vxml:7:5 ->         █    <> SectionTitle
│ example.vxml:8:7            █      <>
│ example.vxml:9:9            █        'Darkness descends'
│ example.vxml:10:5 ->        █    <> Paragraphs
│ example.vxml:11:7           █      <>
│ example.vxml:12:9           █        'This is the third text node'
│ example.vxml:13:9           █        'of the tree, but the first'
│ example.vxml:14:9           █        'text node with >1 lines.'
│ example.vxml:15:7           █      <>
│ example.vxml:16:9           █        'For VXML, this is just a'
│ example.vxml:17:9           █        'second text node. A "paragraph"'
│ example.vxml:18:9           █        'is not one of VXML's abstractions.'
└────────────────────────────────────────────────────────────────────

Import Guide

Most users should start with vxml, vxml/blame, and vxml/io_lines. Use xml_streamer when token-level XML processing is needed.

Tests

Run the package tests from this directory:

gleam test
✨ Search Document