@yozora/tokenizer-paragraph

github flavor markdown spec

A sequence of non-blank lines that cannot be interpreted as other kinds of blocks forms a paragraph. The contents of the paragraph are the result of parsing the paragraph’s raw content as inlines. The paragraph’s raw content is formed by concatenating the lines and removing initial and final whitespace.

• See github flavor markdown spec for details.
• See Live Examples for an intuitive impression.

Install

npm

npm install --save @yozora/tokenizer-paragraph

Yarn

yarn add @yozora/tokenizer-paragraph

pnpm

pnpm add @yozora/tokenizer-paragraph

Usage

tip

@yozora/tokenizer-paragraph has been integrated into @yozora/parser / @yozora/parser-gfm-ex / @yozora/parser-gfm, so you can use YozoraParser / GfmExParser / GfmParser directly.

Basic Usage

@yozora/tokenizer-paragraph cannot be used alone, it needs to be registered in YastParser as a plugin-in before it can be used.

import { DefaultYastParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'

const parser = new DefaultYastParser()
  .useBlockFallbackTokenizer(new ParagraphTokenizer())
  .useInlineFallbackTokenizer(new TextTokenizer())
  .useTokenizer(new ParagraphTokenizer())

// parse source markdown content
parser.parse(`
aaa

bbb
`)

YozoraParser

import YozoraParser from '@yozora/parser'

const parser = new YozoraParser()

// parse source markdown content
parser.parse(`
aaa

bbb
`)

GfmParser

import GfmParser from '@yozora/parser-gfm'

const parser = new GfmParser()

// parse source markdown content
parser.parse(`
aaa

bbb
`)

GfmExParser

import GfmExParser from '@yozora/parser-gfm-ex'

const parser = new GfmExParser()

// parse source markdown content
parser.parse(`
aaa

bbb
`)

Options

Name	Type	Required	Default
name	string	false	"@yozora/tokenizer-paragraph"
priority	number	false	TokenizerPriority.FALLBACK

• name: The unique name of the tokenizer, used to bind the token it generates, to determine the tokenizer that should be called in each life cycle of the token in the entire matching / parsing phase.

• priority: Priority of the tokenizer, determine the order of processing, high priority priority execution. interruptable. In addition, in the match-block stage, a high-priority tokenizer can interrupt the matching process of a low-priority tokenizer.

Types

@yozora/tokenizer-paragraph produce Paragraph type nodes. See @yozora/ast for full base types.

import type { YastParent } from '@yozora/ast'

export const ParagraphType = 'paragraph'
export type ParagraphType = typeof ParagraphType

/**
 * Paragraph represents a unit of discourse dealing with a particular
 * point or idea.
 * @see https://github.com/syntax-tree/mdast#paragraph
 * @see https://github.github.com/gfm/#paragraphs
 */
export type Paragraph = YastParent<ParagraphType>

Live Examples

• Basic.

  #189

  yozora

    

  pretty-json

    

  aaa bbb

  1
  2
  3
  aaa
  
  bbb

• Paragraphs can contain multiple lines, but no blank lines.

  #190

  yozora

    

  pretty-json

    

  aaa bbb ccc ddd

  1
  2
  3
  4
  5
  aaa
  bbb
  
  ccc
  ddd

• Multiple blank lines between paragraph have no effect.

  #191

  yozora

    

  pretty-json

    

  aaa bbb

  1
  2
  3
  4
  aaa
  
  
  bbb

• Leading spaces are skipped.

  #192

  yozora

    

  pretty-json

    

  aaa bbb

  1
  2
    aaa
   bbb

• Lines after the first may be indented any amount, since indented code blocks cannot interrupt paragraphs.

  #193

  yozora

    

  pretty-json

    

  aaa bbb ccc

  1
  2
  3
  aaa
               bbb
                                         ccc

• However, the first line may be indented at most three spaces, or an indented code block will be triggered.

  #194

  yozora

    

  pretty-json

    

  aaa bbb

  1
  2
     aaa
  bbb

• Final spaces are stripped before inline parsing, so a paragraph that ends with two or more spaces will not end with a hard line break.

  #196

  yozora

    

  pretty-json

    

  aaa bbb

  1
  2
  aaa     
  bbb     

Related

• @yozora/ast
• @yozora/parser
• @yozora/parser-gfm
• @yozora/parser-gfm-ex
• @yozora/react-paragraph
• @yozora/react-markdown
• Sourcecode
• Paragraph | Yozora AST
• Paragraphs | Github Flavor Markdown Spec
• Paragraph | Mdast