Version: 1.x.x

@yozora/core-parser

@yozora/core-parser provides a plug-in system that can load tokenizer for collaborative processing in the lifecycles.

Install

npm
Yarn
pnpm

npm install --save @yozora/core-parser

yarn add @yozora/core-parser

pnpm add @yozora/core-parser

Usage

@yozora/core-parser provides a default YastParser implementation DefaultYastParser.

import { DefaultYastParser } from '@yozora/core-parser'
import EmphasisTokenizer from '@yozora/tokenizer-emphasis'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'

const parser = new DefaultYastParser({ shouldReservePosition: true })
  .useBlockFallbackTokenizer(new ParagraphTokenizer())
  .useInlineFallbackTokenizer(new TextTokenizer())

  // add custom tokenizers through `useTokenizer()`
  .useTokenizer(new EmphasisTokenizer())

// Parse content
parser.parse('source content')

// Specify the content range
parser.parse('source content', 0, 10)

// Remove the positions from the YastNode.
parser.parse('source content', undefined, undefined, false)

// Reserve the positions
parser.parse('source content', undefined, undefined, true)

Options

Name	Required	Default	Type
`blockFallbackTokenizer`	`false`	`null`	BlockFallbackTokenizer \| `null`
`inlineFallbackTokenizer`	`false`	`null`	InlineFallbackTokenizer \| `null`
`lazinessTypes`	`false`	See below	`string[]`
`shouldReservePosition`	`false`	`false`	`boolean`

blockFallbackTokenizer: Fallback tokenizer on processing block structure phase.
inlineFallbackTokenizer: Fallback tokenizer on processing inline structure phase.
lazinessTypes: Types of YastNode which could has laziness contents.
- Default: result of [PhrasingContentType].concat(blockFallbackTokenizer.uniqueTypes).
shouldReservePosition: Whether it is necessary to reserve the position of the YastNode parsed.

Lifecycle

match-block

See TokenizerMatchBlockHook for Details.

Name	Required	Type
`isContainerBlock`	`true`	`boolean`
`interruptableTypes`	`true`	`string[]`
`eatOpener`	`true`	`<Function>`
`eatAndInterruptPreviousSibling`	`false`	`<Function>`
`eatContinuationText`	`false`	`<Function>`
`eatLazyContinuationText`	`false`	`<Function>`
`onClose`	`false`	`<Function>`
`extractPhrasingContentLines`	`false`	`<Function>`
`buildBlockState`	`false`	`<Function>`

Examples

post-match-block

See TokenizerPostMatchBlockHook for Details.

Name	Required	Type
`transformMatch`	`true`	`<Function>`

Examples
- @yozora/tokenizer-list

parse-block

See TokenizerParseBlockHook for Details.

Name	Required	Type
`parseBlock`	`true`	`<Function>`
`parseMeta`	`false`	`<Function>`

Examples

match-inline

See TokenizerMatchInlineHook for Details.

Name	Required	Type
`priority`	`number`	`<Function>`
`delimiterGroup`	`string`	`<Function>`
`findDelimiter`	`true`	`<Function>`
`isDelimiterPair`	`false`	`<Function>`
`processDelimiterPair`	`false`	`<Function>`
`processFullDelimiter`	`false`	`<Function>`

Examples

parse-inline

See TokenizerParseInlineHook for Details.

Name	Required	Type
`processToken`	true	`<Function>`

Examples

Install​

Usage​

Options​

Lifecycle​

match-block​

post-match-block​

parse-block​

match-inline​

parse-inline​

Related​