@yozora/tokenizer-break
Hard line breaks
A line break (not in a code span or HTML tag) that is preceded by two or more
spaces and does not occur at the end of a block is parsed as a
hard line break (rendered in HTML as a <br /> tag).
Soft line breaks
A regular line break (not in a code span or HTML tag) that is not preceded by two or more spaces or a backslash is parsed as a [softbreak]gfm-softbreak. (A softbreak may be rendered in HTML either as a line ending or as a space. The result will be the same in browsers. In the examples here, a line ending will be used.)
- See github flavor markdown spec for details.
- See Live Examples for an intuitive impression.
Install
- npm
- Yarn
- pnpm
npm install --save @yozora/tokenizer-break
yarn add @yozora/tokenizer-break
pnpm add @yozora/tokenizer-break
Usage
@yozora/tokenizer-break has been integrated into @yozora/parser / @yozora/parser-gfm-ex / @yozora/parser-gfm,
so you can use YozoraParser / GfmExParser / GfmParser directly.
- Basic Usage
- YozoraParser
- GfmParser
- GfmExParser
@yozora/tokenizer-break cannot be used alone, it needs to be registered in YastParser as a plugin-in before it can be used.
import { DefaultYastParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import BreakTokenizer from '@yozora/tokenizer-break'
const parser = new DefaultYastParser()
.useBlockFallbackTokenizer(new ParagraphTokenizer())
.useInlineFallbackTokenizer(new TextTokenizer())
.useTokenizer(new BreakTokenizer())
// parse source markdown content
parser.parse(`
foo
baz
foo\
baz
foo
baz
`)
import YozoraParser from '@yozora/parser'
const parser = new YozoraParser()
// parse source markdown content
parser.parse(`
foo
baz
foo\
baz
foo
baz
`)
import GfmParser from '@yozora/parser-gfm'
const parser = new GfmParser()
// parse source markdown content
parser.parse(`
foo
baz
foo\
baz
foo
baz
`)
import GfmExParser from '@yozora/parser-gfm-ex'
const parser = new GfmExParser()
// parse source markdown content
parser.parse(`
foo
baz
foo\
baz
foo
baz
`)
Options
| Name | Type | Required | Default |
|---|---|---|---|
name | string | false | "@yozora/tokenizer-break" |
priority | number | false | TokenizerPriority.ATOMIC |
-
name: The unique name of the tokenizer, used to bind the token it generates, to determine the tokenizer that should be called in each life cycle of the token in the entire matching / parsing phase. -
priority: Priority of the tokenizer, determine the order of processing, high priority priority execution. interruptable. In addition, in thematch-blockstage, a high-priority tokenizer can interrupt the matching process of a low-priority tokenizer.Exception: Delimiters of type
fullare always processed before other type delimiters.
Types
@yozora/tokenizer-break produce Break type nodes. See @yozora/ast for full base types.
import type { YastNode } from '@yozora/ast'
export const BreakType = 'break'
export type BreakType = typeof BreakType
/**
* Break represents a line break, such as in poems or addresses.
* @see https://github.com/syntax-tree/mdast#break
* @see https://github.github.com/gfm/#hard-line-breaks
* @see https://github.github.com/gfm/#soft-line-breaks
*/
export type Break = YastNode<BreakType>
Live Examples
Hard line breaks
-
Basic.
-
For a more visible alternative, a backslash before the line ending may be used instead of two spaces.
-
More than two spaces can be used.
-
Leading spaces at the beginning of the next line are ignored.
-
Line breaks can occur inside emphasis, links, and other constructs that allow inline content.
-
Line breaks do not occur inside code spans or or HTML tags.
-
Hard line breaks are for separating inline content within a block. Neither syntax for hard line breaks works at the end of a paragraph or other block element.
Soft line breaks
-
Basic.
-
Spaces at the end of the line and beginning of the next line are removed.