@yozora/tokenizer-setext-heading

github flavor markdown spec

A setext heading consists of one or more lines of text, each containing at least one non-whitespace character, with no more than $3$ spaces indentation, followed by a setext heading underline. The lines of text must be such that, were they not followed by the setext heading underline, they would be interpreted as a paragraph: they cannot be interpretable as a code fence, ATX heading, block quote, thematic break, list item, or HTML block.

A setext heading underline is a sequence of = characters or a sequence of - characters, with no more than $3$ spaces indentation and any number of trailing spaces. If a line containing a single - can be interpreted as an empty list items, it should be interpreted this way and not as a setext heading underline.

The heading is a level $1$ heading if = characters are used in the setext heading underline, and a level $2$ heading if - characters are used. The contents of the heading are the result of parsing the preceding lines of text as CommonMark inline content.

In general, a setext heading need not be preceded or followed by a blank line. However, it cannot interrupt a paragraph, so when a setext heading comes after a paragraph, a blank line is needed between them.

• See github flavor markdown spec for details.
• See Live Examples for an intuitive impression.

Install

npm

npm install --save @yozora/tokenizer-setext-heading

Yarn

yarn add @yozora/tokenizer-setext-heading

pnpm

pnpm add @yozora/tokenizer-setext-heading

Usage

tip

@yozora/tokenizer-setext-heading has been integrated into @yozora/parser / @yozora/parser-gfm-ex / @yozora/parser-gfm, so you can use YozoraParser / GfmExParser / GfmParser directly.

Basic Usage

@yozora/tokenizer-setext-heading cannot be used alone, it needs to be registered in YastParser as a plugin-in before it can be used.

import { DefaultYastParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import SetextHeadingTokenizer from '@yozora/tokenizer-setext-heading'

const parser = new DefaultYastParser()
  .useBlockFallbackTokenizer(new ParagraphTokenizer())
  .useInlineFallbackTokenizer(new TextTokenizer())
  .useTokenizer(new SetextHeadingTokenizer())

// parse source markdown content
parser.parse(`
Foo *bar*
=========

Foo *bar*
---------
`)

YozoraParser

import YozoraParser from '@yozora/parser'

const parser = new YozoraParser()

// parse source markdown content
parser.parse(`
Foo *bar*
=========

Foo *bar*
---------
`)

GfmParser

import GfmParser from '@yozora/parser-gfm'

const parser = new GfmParser()

// parse source markdown content
parser.parse(`
Foo *bar*
=========

Foo *bar*
---------
`)

GfmExParser

import GfmExParser from '@yozora/parser-gfm-ex'

const parser = new GfmExParser()

// parse source markdown content
parser.parse(`
Foo *bar*
=========

Foo *bar*
---------
`)

Options

Name	Type	Required	Default
name	string	false	"@yozora/tokenizer-setext-heading"
priority	number	false	TokenizerPriority.ATOMIC

• name: The unique name of the tokenizer, used to bind the token it generates, to determine the tokenizer that should be called in each life cycle of the token in the entire matching / parsing phase.

• priority: Priority of the tokenizer, determine the order of processing, high priority priority execution. interruptable. In addition, in the match-block stage, a high-priority tokenizer can interrupt the matching process of a low-priority tokenizer.

Types

@yozora/tokenizer-setext-heading produce Heading type nodes. See @yozora/ast for full base types.

import type { YastParent } from '@yozora/ast'

export const HeadingType = 'heading'
export type HeadingType = typeof HeadingType

/**
 * Heading represents a heading of a section.
 * @see https://github.com/syntax-tree/mdast#heading
 * @see https://github.github.com/gfm/#atx-heading
 */
export interface Heading extends YastParent<HeadingType> {
  /**
   * HTML anchor identifier.
   */
  identifier?: string
  /**
   * level of heading
   */
  depth: 1 | 2 | 3 | 4 | 5 | 6
}

Live Examples

• Basic.

  #50

  yozora

    

  pretty-json

    

  Foo *bar* ========= Foo *bar* ---------

  1
  2
  3
  4
  5
  Foo *bar*
  =========
  
  Foo *bar*
  ---------

• The content of the header may span more than one line.

  #51

  yozora

    

  pretty-json

    

  Foo *bar baz* ====

  1
  2
  3
  Foo *bar
  baz*
  ====

• The contents are the result of parsing the headings’s raw content as inlines. The heading’s raw content is formed by concatenating the lines and removing initial and final whitespace.

  #52

  yozora

    

  pretty-json

    

  Foo *bar baz* ====

  1
  2
  3
    Foo *bar
  baz*	
  ====

• The underlining can be any length

  #53

  yozora

    

  pretty-json

    

  Foo ------------------------- Foo =

  1
  2
  3
  4
  5
  Foo
  -------------------------
  
  Foo
  =

• The heading content can be indented up to three spaces, and need not line up with the underlining.

  #54

  yozora

    

  pretty-json

    

  Foo --- Foo ----- Foo ===

  1
  2
  3
  4
  5
  6
  7
  8
     Foo
  ---
  
    Foo
  -----
  
    Foo
    ===

• Four spaces indent is too much

  #55

  yozora

    

  pretty-json

    

  Foo --- Foo ---

  1
  2
  3
  4
  5
      Foo
      ---
  
      Foo
  ---

• The setext heading underline can be indented up to three spaces, and may have trailing spaces.

  #56

  yozora

    

  pretty-json

    

  Foo ----

  1
  2
  Foo
     ----      

• The setext heading underline cannot contain internal spaces.

  #58

  yozora

    

  pretty-json

    

  Foo = = Foo --- -

  1
  2
  3
  4
  5
  Foo
  = =
  
  Foo
  --- -

• Trailing spaces in the content line do not cause a line break.

  #59

  yozora

    

  pretty-json

    

  Foo -----

  1
  2
  Foo  
  -----

• Nor does a backslash at the end.

  #60

  yozora

    

  pretty-json

    

  Foo\ ----

  1
  2
  Foo\
  ----

• Since indicators of block structure take precedence over indicators of inline structure, the following are setext headings.

  #61

  yozora

    

  pretty-json

    

  `Foo ---- ` <a title="a lot --- of dashes"/>

  1
  2
  3
  4
  5
  6
  7
  `Foo
  ----
  `
  
  <a title="a lot
  ---
  of dashes"/>

• The setext heading underline cannot be a lazy continuation line in a list item or block quote.

  #62

  yozora

    

  pretty-json

    

  > Foo ---

  1
  2
  > Foo
  ---

• A blank line is needed between a paragraph and a following setext heading, since otherwise the paragraph becomes part of the heading’s content.

  #65

  yozora

    

  pretty-json

    

  Foo Bar ---

  1
  2
  3
  Foo
  Bar
  ---

• But in general a blank line is not required before or after setext headings.

  #66

  yozora

    

  pretty-json

    

  --- Foo --- Bar --- Baz

  1
  2
  3
  4
  5
  6
  ---
  Foo
  ---
  Bar
  ---
  Baz

• Setext headings cannot be empty.

  #67

  yozora

    

  pretty-json

    

  ====

  1
  2
  
  ====

• Setext heading text lines must not be interpretable as block constructs other than paragraphs. So, the line of dashes in these examples gets interpreted as a thematic break.

  #68

  yozora

    

  pretty-json

    

  --- ---

  1
  2
  ---
  ---

• If you want a heading with > foo as its literal text, you can use backslash escapes.

  #72

  yozora

    

  pretty-json

    

  \> foo ------

  1
  2
  \> foo
  ------

• Authors who want interpretation 2 can put blank lines around the thematic break or use a thematic break that cannot count as a setext heading underline.

  #74

  yozora

    

  pretty-json

    

  Foo bar --- baz

  1
  2
  3
  4
  5
  6
  Foo
  bar
  
  ---
  
  baz

• Authors who want interpretation 3 can use backslash escapes.

  #76

  yozora

    

  pretty-json

    

  Foo bar \--- baz

  1
  2
  3
  4
  Foo
  bar
  \---
  baz

Related

• @yozora/ast
• @yozora/parser
• @yozora/parser-gfm
• @yozora/parser-gfm-ex
• @yozora/tokenizer-heading
• @yozora/react-heading
• @yozora/react-markdown
• Sourcecode
• Heading | Yozora AST
• Setext headings | Github Flavor Markdown Spec
• Heading | Mdast