@yozora/tokenizer-indented-code
An indented code block is composed of one or more indented chunks separated by blank lines. An indented chunk is a sequence of non-blank lines, each indented four or more spaces. The contents of the code block are the literal contents of the lines, including trailing line endings, minus four spaces of indentation. An indented code block has no info string.
- See github flavor markdown spec for details.
- See Live Examples for an intuitive impression.
Install
- npm
- Yarn
- pnpm
npm install --save @yozora/tokenizer-indented-code
yarn add @yozora/tokenizer-indented-code
pnpm add @yozora/tokenizer-indented-code
Usage
@yozora/tokenizer-indented-code has been integrated into @yozora/parser / @yozora/parser-gfm-ex / @yozora/parser-gfm,
so you can use YozoraParser
/ GfmExParser
/ GfmParser
directly.
- Basic Usage
- YozoraParser
- GfmParser
- GfmExParser
@yozora/tokenizer-indented-code cannot be used alone, it needs to be registered in YastParser as a plugin-in before it can be used.
import { DefaultYastParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import IndentedCodeTokenizer from '@yozora/tokenizer-indented-code'
const parser = new DefaultYastParser()
.useBlockFallbackTokenizer(new ParagraphTokenizer())
.useInlineFallbackTokenizer(new TextTokenizer())
.useTokenizer(new IndentedCodeTokenizer())
// parse source markdown content
parser.parse(`
<a/>
*hi*
- one
`)
import YozoraParser from '@yozora/parser'
const parser = new YozoraParser()
// parse source markdown content
parser.parse(`
<a/>
*hi*
- one
`)
import GfmParser from '@yozora/parser-gfm'
const parser = new GfmParser()
// parse source markdown content
parser.parse(`
<a/>
*hi*
- one
`)
import GfmExParser from '@yozora/parser-gfm-ex'
const parser = new GfmExParser()
// parse source markdown content
parser.parse(`
<a/>
*hi*
- one
`)
Options
Name | Type | Required | Default |
---|---|---|---|
name | string | false | "@yozora/tokenizer-indented-code" |
priority | number | false | TokenizerPriority.ATOMIC |
-
name
: The unique name of the tokenizer, used to bind the token it generates, to determine the tokenizer that should be called in each life cycle of the token in the entire matching / parsing phase. -
priority
: Priority of the tokenizer, determine the order of processing, high priority priority execution. interruptable. In addition, in thematch-block
stage, a high-priority tokenizer can interrupt the matching process of a low-priority tokenizer.
Types
@yozora/tokenizer-indented-code produce Code type nodes. See @yozora/ast for full base types.
import type { YastLiteral } from '@yozora/ast'
export const CodeType = 'code'
export type CodeType = typeof CodeType
/**
* Code represents a block of preformatted text, such as ASCII art or computer
* code.
* @see https://github.com/syntax-tree/mdast#code
* @see https://github.github.com/gfm/#code-fence
*/
export interface Code extends YastLiteral<CodeType> {
/**
* Language of the codes
*/
lang?: string
/**
* Meta info string
*/
meta?: string
}
Live Examples
-
Basic.
-
If there is any ambiguity between an interpretation of indentation as a code block and as indicating that material belongs to a list item, the list item interpretation takes precedence.
-
The contents of a code block are literal text, and do not get parsed as Markdown.
-
Here we have three chunks separated by blank lines.
-
Any initial spaces beyond four will be included in the content, even in interior blank lines.
-
An indented code block cannot interrupt a paragraph. (This allows hanging indents and the like.)
-
However, any non-blank line with fewer than four leading spaces ends the code block immediately. So a paragraph may occur immediately after indented code.
-
And indented code can occur immediately before and after other kinds of blocks.
-
The first line can be indented more than four spaces.
-
Blank lines preceding or following an indented code block are not included in it.
-
Trailing spaces are included in the code block’s content.