medium severityCrawl4AI (AsyncWebCrawler, /crawl endpoint)
Crawl4AI misses dynamic/lazy-loaded content (e.g., titles, load-more items), extracts partial/wrong page HTML/markdown. Docker /crawl gets parent page content. Session reuse throws "list index out of range". Python SDK often succeeds where endpoints fail.
Root cause
Incomplete page rendering/scanning in Docker /crawl endpoint and session reuse: browser context.pages empty during js_only multi-step crawls, missing dynamic content until full scan/scroll triggered. Browser manager assumes non-empty pages list.
Crawl4AIdynamic contentscan_full_pageDockerjs_onlysession_idlist index out of rangeinfinite scroll