@yozora/tokenizer-list
A list is a sequence of one or more list items of the same type. The list items may be separated by any number of blank lines.
Two list items are of the same type if they begin with a list marker of the same type. Two list markers are of the same type if
-
(a) they are bullet list markers using the same character (
-
,+
, or*
) or -
(b) they are ordered list numbers with the same delimiter (either
.
or)
).
A list is an ordered list if its constituent list items begin with ordered list markers, and a bullet list if its constituent list items begin with bullet list markers.
The start number of an ordered list is determined by the list number of its initial list item. The numbers of subsequent list items are disregarded.
A list is loose if any of its constituent list items are separated
by blank lines, or if any of its constituent list items
directly contain two block-level elements with a blank line
between them. Otherwise a list is tight. (The difference in HTML
output is that paragraphs in a loose list are wrapped in <p>
tags, while
paragraphs in a tight list are not.)
- See github flavor markdown spec for details.
- See Live Examples for an intuitive impression.
Install
- npm
- Yarn
- pnpm
npm install --save @yozora/tokenizer-list
yarn add @yozora/tokenizer-list
pnpm add @yozora/tokenizer-list
Usage
@yozora/tokenizer-list has been integrated into @yozora/parser / @yozora/parser-gfm-ex / @yozora/parser-gfm,
so you can use YozoraParser
/ GfmExParser
/ GfmParser
directly.
- Basic Usage
- YozoraParser
- GfmParser
- GfmExParser
@yozora/tokenizer-list cannot be used alone, it needs to be registered in YastParser as a plugin-in before it can be used.
import { DefaultYastParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import ListTokenizer from '@yozora/tokenizer-list'
const parser = new DefaultYastParser()
.useBlockFallbackTokenizer(new ParagraphTokenizer())
.useInlineFallbackTokenizer(new TextTokenizer())
.useTokenizer(new ListTokenizer())
// parse source markdown content
parser.parse(`
- a
- b
- c
- d
- e
- f
- g
---
- [ ] This is a TODO item.
- [-] This is a processing TODO item.
- [x] This is a finished TODO item.
---
1. This is an ordered list item
a. This is an another type of ordered list item
`)
import YozoraParser from '@yozora/parser'
const parser = new YozoraParser()
// parse source markdown content
parser.parse(`
- a
- b
- c
- d
- e
- f
- g
---
- [ ] This is a TODO item.
- [-] This is a processing TODO item.
- [x] This is a finished TODO item.
---
1. This is an ordered list item
a. This is an another type of ordered list item
`)
import GfmParser from '@yozora/parser-gfm'
const parser = new GfmParser()
// parse source markdown content
parser.parse(`
- a
- b
- c
- d
- e
- f
- g
---
- [ ] This is a TODO item.
- [-] This is a processing TODO item.
- [x] This is a finished TODO item.
---
1. This is an ordered list item
a. This is an another type of ordered list item
`)
import GfmExParser from '@yozora/parser-gfm-ex'
const parser = new GfmExParser()
// parse source markdown content
parser.parse(`
- a
- b
- c
- d
- e
- f
- g
---
- [ ] This is a TODO item.
- [-] This is a processing TODO item.
- [x] This is a finished TODO item.
---
1. This is an ordered list item
a. This is an another type of ordered list item
`)
Options
Name | Type | Required | Default |
---|---|---|---|
name | string | false | "@yozora/tokenizer-list" |
priority | number | false | TokenizerPriority.CONTAINING_BLOCK |
-
name
: The unique name of the tokenizer, used to bind the token it generates, to determine the tokenizer that should be called in each life cycle of the token in the entire matching / parsing phase. -
priority
: Priority of the tokenizer, determine the order of processing, high priority priority execution. interruptable. In addition, in thematch-block
stage, a high-priority tokenizer can interrupt the matching process of a low-priority tokenizer.
Types
@yozora/tokenizer-list produce List type nodes. See @yozora/ast for full base types.
import type { ListItem, YastParent } from '@yozora/ast'
export const ListType = 'list'
export type ListType = typeof ListType
/**
* List represents a list of items.
* @see https://github.com/syntax-tree/mdast#list
* @see https://github.github.com/gfm/#list
*/
export interface List extends YastParent<ListType> {
/**
* Whether it is an ordered lit.
*/
ordered: boolean
/**
* The starting number of a ordered list-item.
*/
start?: number
/**
* Marker of a unordered list-item, or delimiter of an ordered list-item.
*/
marker: number
/**
* Whether if the list is loose.
* @see https://github.github.com/gfm/#loose
*/
spread: boolean
/**
* Lists are container block.
*/
children: ListItem[]
}
Live Examples
-
Basic.
-
In order to solve of unwanted lists in paragraphs with hard-wrapped numerals, we allow only lists starting with to interrupt paragraphs.
-
There can be any number of blank lines between items.
-
To separate consecutive lists of the same type, or to separate a list from an indented code block that would otherwise be parsed as a subparagraph of the final list item, you can insert a blank HTML comment.
-
List items need not be indented to the same level. The following list items will be treated as items at the same list level, since none is indented enough to belong to the previous list item.
-
Note, however, that list items may not be indented more than three spaces. Here
- e
is treated as a paragraph continuation line, because it is indented more than three spaces. -
And here,
3. c
is treated as in indented code block, because it is indented four spaces and preceded by a blank line. -
This is a loose list, because there is a blank line between two of the list items.
-
So is this, with a empty second item.
-
These are loose lists, even though there is no space between the items, because one of the items directly contains two block-level elements with a blank line between them.
-
This is a tight list, because the blank lines are in a code block.
-
This is a tight list, because the blank line is between two paragraphs of a sublist. So the sublist is loose while the outer list is tight.
-
This is a tight list, because the blank line is inside the block quote.
-
This list is tight, because the consecutive block elements are not separated by blank lines.
-
A single-paragraph list is tight.
-
This list is loose, because of the blank line between the two block elements in the list item.
-
Here the outer list is loose, the inner list tight.