@yozora/tokenizer-math

note

Install

npm

npm install --save @yozora/tokenizer-math

Yarn

yarn add @yozora/tokenizer-math

pnpm

pnpm add @yozora/tokenizer-math

Usage

tip

@yozora/tokenizer-math has been integrated into @yozora/parser, so you can use YozoraParser directly.

Basic Usage

@yozora/tokenizer-math cannot be used alone, it needs to be registered in Parser as a plugin-in before it can be used.

import { DefaultParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import MathTokenizer from '@yozora/tokenizer-math'

const parser = new DefaultParser()
  .useFallbackTokenizer(new ParagraphTokenizer())
  .useFallbackTokenizer(new TextTokenizer())
  .useTokenizer(new MathTokenizer())

// parse source markdown content
parser.parse(`
$$x^2 + y^2=z^2$$

$$
f(x)=\left\lbrace\begin{aligned}
&x^2, &x < 0\\
&0, &x = 0\\
&x^3, &x > 0
\end{aligned}\right.
$$
`)

YozoraParser

import YozoraParser from '@yozora/parser'

const parser = new YozoraParser()

// parse source markdown content
parser.parse(`
$$x^2 + y^2=z^2$$

$$
f(x)=\left\lbrace\begin{aligned}
&x^2, &x < 0\\
&0, &x = 0\\
&x^3, &x > 0
\end{aligned}\right.
$$
`)

GfmParser

import GfmParser from '@yozora/parser-gfm'
import MathTokenizer from '@yozora/tokenizer-math'

const parser = new GfmParser()
parser.useTokenizer(new MathTokenizer())

// parse source markdown content
parser.parse(`
$$x^2 + y^2=z^2$$

$$
f(x)=\left\lbrace\begin{aligned}
&x^2, &x < 0\\
&0, &x = 0\\
&x^3, &x > 0
\end{aligned}\right.
$$
`)

GfmExParser

import GfmExParser from '@yozora/parser-gfm-ex'
import MathTokenizer from '@yozora/tokenizer-math'

const parser = new GfmExParser()
parser.useTokenizer(new MathTokenizer())

// parse source markdown content
parser.parse(`
$$x^2 + y^2=z^2$$

$$
f(x)=\left\lbrace\begin{aligned}
&x^2, &x < 0\\
&0, &x = 0\\
&x^3, &x > 0
\end{aligned}\right.
$$
`)

Options

Name	Type	Required	Default
name	string	false	"@yozora/tokenizer-math"
priority	number	false	TokenizerPriority.FENCED_BLOCK

• name: The unique name of the tokenizer, used to bind the token it generates, to determine the tokenizer that should be called in each life cycle of the token in the entire matching / parsing phase.

• priority: Priority of the tokenizer, determine the order of processing, high priority priority execution. interruptable. In addition, in the match-block stage, a high-priority tokenizer can interrupt the matching process of a low-priority tokenizer.

Types

@yozora/tokenizer-math produce Math type nodes. See @yozora/ast for full base types.

import type { Literal } from '@yozora/ast'

export const MathType = 'math'
export type MathType = typeof MathType

/**
 * Math content.
 */
export type Math = Literal<MathType>

Live Examples

• Single line math block.

  #1

  yozora

    

  pretty-json

    

  $$x^2 + y^2=z^2$$

  1
  $$x^2 + y^2=z^2$$

• Multiple line math block.

  #5

  yozora

    

  pretty-json

    

  $$ f(x)=\left\lbrace\begin{aligned} &x^2, &x < 0\\ &0, &x = 0\\ &x^3, &x > 0 \end{aligned}\right. $$

  1
  2
  3
  4
  5
  6
  7
  $$
  f(x)=\left\lbrace\begin{aligned}
  &x^2, &x < 0\\
  &0, &x = 0\\
  &x^3, &x > 0
  \end{aligned}\right.
  $$

Related

• @yozora/ast
• @yozora/parser
• @yozora/parser-gfm
• @yozora/parser-gfm-ex
• @yozora/tokenizer-inline-math
• @yozora/react-math
• @yozora/react-markdown
• Sourcecode
• Math | Yozora AST