@yozora/tokenizer-inline-code
A backtick string is a string of one or more backtick
characters (`) that is neither preceded nor followed by a backtick.
A code span begins with a backtick string and ends with a backtick string of equal length. The contents of the code span are the characters between the two backtick strings, normalized in the following ways:
-
First, line endings are converted to spaces.
-
If the resulting string both begins and ends with a space character, but does not consist entirely of space characters, a single space character is removed from the front and back. This allows you to include code that begins or ends with backtick characters, which must be separated by whitespace from the opening or closing backtick strings.
- See github flavor markdown spec for details.
- See Live Examples for an intuitive impression.
Install
- npm
- Yarn
- pnpm
npm install --save @yozora/tokenizer-inline-code
yarn add @yozora/tokenizer-inline-code
pnpm add @yozora/tokenizer-inline-code
Usage
@yozora/tokenizer-inline-code has been integrated into @yozora/parser / @yozora/parser-gfm-ex / @yozora/parser-gfm,
so you can use YozoraParser / GfmExParser / GfmParser directly.
- Basic Usage
- YozoraParser
- GfmParser
- GfmExParser
@yozora/tokenizer-inline-code cannot be used alone, it needs to be registered in YastParser as a plugin-in before it can be used.
import { DefaultYastParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import InlineCodeTokenizer from '@yozora/tokenizer-inline-code'
const parser = new DefaultYastParser()
.useBlockFallbackTokenizer(new ParagraphTokenizer())
.useInlineFallbackTokenizer(new TextTokenizer())
.useTokenizer(new InlineCodeTokenizer())
// parse source markdown content
parser.parse("`inline code`")
import YozoraParser from '@yozora/parser'
const parser = new YozoraParser()
// parse source markdown content
parser.parse("`inline code`")
import GfmParser from '@yozora/parser-gfm'
const parser = new GfmParser()
// parse source markdown content
parser.parse("`inline code`")
import GfmExParser from '@yozora/parser-gfm-ex'
const parser = new GfmExParser()
// parse source markdown content
parser.parse("`inline code`")
Options
| Name | Type | Required | Default |
|---|---|---|---|
name | string | false | "@yozora/tokenizer-inline-code" |
priority | number | false | TokenizerPriority.ATOMIC |
-
name: The unique name of the tokenizer, used to bind the token it generates, to determine the tokenizer that should be called in each life cycle of the token in the entire matching / parsing phase. -
priority: Priority of the tokenizer, determine the order of processing, high priority priority execution. interruptable. In addition, in thematch-blockstage, a high-priority tokenizer can interrupt the matching process of a low-priority tokenizer.Exception: Delimiters of type
fullare always processed before other type delimiters.
Types
@yozora/tokenizer-inline-code produce InlineCode type nodes. See @yozora/ast for full base types.
import type { YastLiteral } from '@yozora/ast'
export const InlineCodeType = 'inlineCode'
export type InlineCodeType = typeof InlineCodeType
/**
* InlineCode represents a fragment of computer code, such as a file name,
* computer program, or anything a computer could parse.
* @see https://github.com/syntax-tree/mdast#inline-code
* @see https://github.github.com/gfm/#code-span
*/
export type InlineCode = YastLiteral<InlineCodeType>
Live Examples
-
Basic.
-
Here two backticks are used, because the code contains a backtick. This example also illustrates stripping of a single leading and trailing space.
-
This example shows the motivation for stripping leading and trailing spaces.
-
Note that only one space is stripped.
-
The stripping only happens if the space is on both sides of the string.
-
Only spaces, and not unicode whitespace in general, are stripped in this way.
-
No stripping occurs if the code span contains only spaces.
-
Line endings are treated like spaces.
-
Interior spaces are not collapsed.
-
Note that backslash escapes do not work in code spans. All backslashes are treated literally.
-
Backslash escapes are never needed, because one can always choose a string of backtick characters as delimiters, where the code does not contain any strings of exactly backtick characters.
-
Code span backticks have higher precedence than any other inline constructs except HTML tags and autolinks. Thus, for example, this is not parsed as emphasized text, since the second
*is part of a code span. -
And this is not parsed as a link.
-
Code spans, HTML tags, and autolinks have the same precedence. Thus, this is code span.
-
But this is an HTML tag.
-
And this is code.
-
But this is an autolink.
-
When a backtick string is not closed by a matching backtick string, we just have literal backticks.
-
The following case also illustrates the need for opening and closing backtick strings to be equal in length.