WikitextParser

WikitextParser is a dependency-free .NET 9 library for parsing Wikipedia's wikitext markup into a structured object model. It provides both a low-level API for direct element access and a high-level API for parsing a full page into a semantic structure of sections, subsections, and an infobox.

The entire library was developed to parse real-world Wikipedia page and can handle a variety of common wikitext syntax.

Features

• Dual-Level API:
  • High-Level API (Parser.ParsePage): Parses an entire page into a structured Page object with a lead section, an infobox, and a nested hierarchy of sections.
  • Low-Level API (Parser.Parse): Parses wikitext into a flat IEnumerable<WikitextElement> for custom processing and analysis.
• Content Conversion: Convert any parsed element, section, or the entire page into either HTML or plain text.
• No Dependencies: The core library is self-contained and does not require any external packages.

Installation

This library will be available via NuGet. You can install it using the .NET CLI:

dotnet add package WikitextParser

Or through the NuGet Package Manager in Visual Studio.

Usage Guide

Using the parser is straightforward. The recommended approach is to use the high-level ParsePage API for most use cases.

High-Level API: Parsing a Page

This is the easiest way to get a structured representation of a wiki page.

using WikitextParser; using WikitextParser.Models; string wikitext = File.ReadAllText("my_wikitext_file.txt"); // 1. Parse the entire page into a Page object Page page = Parser.ParsePage(wikitext); // 2. Access the Infobox if (page.Infobox != null) { var creatorParam = page.Infobox.Parameters.FirstOrDefault(p => p.Key == "creator"); if (creatorParam != null) { Console.WriteLine($"Creator: {creatorParam.Value.ConvertToText()}"); } } // 3. Access the Lead Content (content before the first heading) foreach (var element in page.LeadContent) { // Ignores metadata templates like {{Short description|...}} if (element is not WikiKeyValuePairElement) { Console.WriteLine(element.ConvertToText()); } } // 4. Recursively process all sections and subsections void PrintSection(Section section, int indent = 0) { string indentStr = new string(' ', indent * 2); Console.WriteLine($"{indentStr}{section.Heading.ConvertToText().Trim()}"); // Print main article links for the section foreach (var mainLink in section.MainArticleLinks) { Console.WriteLine($"{indentStr} (See also: {mainLink.Parameters.First().Value.ConvertToText()})"); } // Print the section's direct content foreach(var content in section.ContentElements) { Console.WriteLine($"{indentStr} [Content: {content.GetType().Name}]"); } // Recurse into subsections foreach (var sub in section.Subsections) { PrintSection(sub, indent + 1); } } Console.WriteLine("\n--- PAGE STRUCTURE ---"); foreach (var section in page.Sections) { PrintSection(section); } // 5. Convert the entire page to HTML or Text string fullHtml = page.ConvertToHtml(); string fullText = page.ConvertToText();