URL Import

Importing from URLs

Fetch and import content directly from web pages using their URLs.

Step-by-Step Process

1. Navigate to Components > HTML Import Pro > Import
2. Click From URL
3. Enter one or more URLs (one per line)
4. Click Add URLs to queue them
5. Configure extraction and article settings
6. Click Start Import

AJAX-Based Processing

URL imports use AJAX for real-time feedback:

• Progress bar shows overall completion
• Current URL being processed is displayed
• Each URL shows status (pending, fetching, completed, error)
• Import continues even if individual URLs fail

URL Import Settings

• Fetch Interval: Delay between requests (prevents rate limiting)
• User Agent: Browser identification string
• Request Timeout: Maximum wait time per URL
• Respect robots.txt: Honor website crawling rules

Automatic Alias Generation

Article aliases are automatically generated from URLs:

• https://example.com/my-article/ → my-article
• Duplicate aliases get numeric suffixes (my-article-2)

Handling Errors

Common issues and solutions:

• 403 Forbidden: Website may block bots. Try different User Agent.
• Timeout: Increase request timeout in settings
• SSL Error: Server may have certificate issues
• Firewall Block: Your server's firewall may block outgoing requests