Crawl4AI Plus Advanced icon

Crawl4AI Plus Advanced

Advanced web crawling and extraction with full Crawl4AI API control

Actions15

SEO Metadata Extractor

AI-generated

Summary

Extract SEO-related metadata from a specified webpage URL, including configurable types such as basic meta tags, JSON-LD structured data, language/locale info, Open Graph tags, robots directives, and Twitter card metadata.

Inputs

  • url (required) — The URL of the webpage to extract SEO metadata from.
  • metadataTypes — An array selecting which types of SEO metadata to extract. Options include Basic Meta Tags, JSON-LD Structured Data, Language & Locale, Open Graph Tags, Robots & Indexing directives, and Twitter Cards. Defaults to ['basic','openGraph','jsonLd'].
  • options.includeFullText — Boolean to include the original full webpage text in the output.
  • options.includeRawHtml — Boolean to include the raw HTML head section in the output.
  • browserSession — Collection of browser and session configuration options such as browser profile, engine type, user agent, headers, JavaScript enabling, stealth mode, proxy, cookies, viewport size, timeout, and other settings to control how the webpage is fetched and rendered.
  • crawlSettings — Collection of crawl settings affecting how the page is loaded and content extracted, including anti-bot measures, resource blocking (ads, CSS), cache mode, robots.txt compliance, content filtering selectors, wait conditions before extraction, retry limits, and JavaScript code injections.

Output shape

a single record containing extracted SEO metadata fields as per the selected metadataTypes, optionally including original page text and raw HTML head if requested.

The output fields vary depending on the selected metadata types; extracted data includes structured information such as title, description, keywords, JSON-LD data, language tags, Open Graph tags, robots directives, and Twitter cards. Additional metadata like the original full text and raw HTML head section are optional. The operation uses a configurable headless browser session with advanced crawl settings to handle page rendering and anti-bot protections. Errors during extraction may be surfaced if extraction fails.

Links

Discussion