Loading page
This workflow contains community nodes that are only compatible with the self-hosted version of n8n. This workflow automatically monitors keyword rankings across search engines to track SEO performance and identify optimization opportunities. It saves you time by eliminating the need to manually check keyword positions and provides comprehensive ranking data for strategic SEO decision making. Overview This workflow automatically scrapes search engine results pages (SERPs) to track keyword rankings, competitor positions, and search features. It uses Bright Data to access search results without restrictions and AI to intelligently parse ranking data, track changes, and identify SEO opportunities. Tools Used n8n: The automation platform that orchestrates the workflow Bright Data: For scraping search engine results without being blocked OpenAI: AI agent for intelligent ranking analysis and SEO insights Google Sheets: For storing keyword ranking data and tracking changes How to Install 1. Import the Workflow: Download the .json file and import it into your n8n instance 2. Configure Bright Data: Add your Bright Data credentials to the MCP Client node 3. Set Up OpenAI: Configure your OpenAI API credentials 4. Configure Google Sheets: Connect your Google Sheets account and set up your ranking tracking spreadsheet 5. Customize: Define target keywords and ranking monitoring parameters Use Cases SEO Teams: Track keyword performance and identify ranking improvements Content Marketing: Monitor content ranking success and optimization needs Competitive Analysis: Track competitor keyword rankings and strategies Digital Marketing: Measure organic search performance and ROI Connect with Me Website: https://www.nofluff.online YouTube: https://www.youtube.com/@YaronBeen/videos LinkedIn: https://www.linkedin.com/in/yaronbeen/ Get Bright Data: https://get.brightdata.com/1tndi4600b25 (Using this link supports my free workflows with a small commission) #n8n #automation #keywordrankings #seo #searchrankings #brightdata #webscraping #seotools #n8nworkflow #workflow #nocode #ranktracking #keywordmonitoring #seoautomation #searchmarketing #organicseo #seoresearch #rankinganalysis #keywordanalysis #searchengines #seomonitoring #digitalmarketing #serp #keywordtracking #seoanalytics #searchoptimization #rankingreports #keywordresearch #seoinsights #searchperformance
by Yaron BeenDescription This workflow automates the process of scraping the latest discussions from HackerNews, transforming raw threads into human readable content using Google Gemini, and exporting the final content into a well-formatted Google Doc. Overview This n8n workflow is responsible for extracting trending posts from the HackerNews API. It loops through each item, performs HTTP data extraction, utilizes Google Gemini to generate human-readable insights, and then exports the enriched content into Google Docs for distribution, archiving, or content creation. Who this workflow is for Tech Newsletter Writers: Automate the collection and summarization of trending HackerNews posts for inclusion in weekly or daily newsletters. Content Creators & Bloggers: Quickly generate structured summaries and insights from HackerNews threads to use as inspiration or supporting content for blog posts, videos, or social media. Startup Founders & Product Builders: Monitor HackerNews for discussions relevant to your niche or competitors, and keep a pulse on the community’s opinions. Investors & Analysts: Surface early signals from the tech ecosystem by identifying what’s trending and how the community is reacting. Researchers & Students: Analyze popular discussions and emerging trends in technology, programming, and startups—enriched with AI-generated insights. Digital Agencies & Consultants: Offer HackerNews monitoring and insight reports as a value-added service to clients interested in the tech space. Tools Used n8n: The core automation engine that manages the trigger, transformation, and export. HackerNews API: Provides access to trending or new HN posts. Google Gemini: Enriches HackerNews content with structured insights and human-like summaries. Google Docs: Automatically creates and updates a document with the enriched content, ready for sharing or publishing. How to Install Import the Workflow: Download the .json file and import it into your n8n instance. Set Up HackerNews Source: Choose whether to use the HN API (via HTTP Request node) or RSS Feed node. Configure Gemini API: Add your Google Gemini API key and design the prompt to extract pros/cons, key themes, or insights. Set Up Google Docs Integration: Connect your Google account and configure the Google Docs node to create/update a document. Test and Deploy: Run a test job to ensure data flows correctly and outputs are formatted as expected. Use Cases Tech Newsletter Authors: Generate ready-to-use summaries of trending HackerNews threads. Startup Founders: Stay informed on key discussions, product launches, and community feedback. Investors & Analysts: Spot early trends, technical insights, and startup momentum directly from HN. Researchers: Track community reactions to new technologies or frameworks. Content Creators: Use the enriched data to spark blog posts, YouTube scripts, or LinkedIn updates. Connect with Me Email: ranjancse@gmail.com LinkedIn: https://www.linkedin.com/in/ranjan-dailata/ Get Bright Data: Bright Data (Supports free workflows with a small commission) #n8n #automation #hackernews #contentcuration #aiwriting #geminiapi #googlegemini #techtrends #newsletterautomation #googleworkspace #rssautomation #nocode #structureddata #webscraping #contentautomation #hninsights #aiworkflow #googleintegration #webmonitoring #hnnews #aiassistant #gdocs #automationtools #gptlike #geminiwriter
by Ranjan DailataThis workflow contains community nodes that are only compatible with the self-hosted version of n8n. This workflow automatically monitors competitor pricing changes and website updates to keep you informed of market movements. It saves you time by eliminating the need to manually check competitor websites and provides alerts only when actual changes occur, preventing information overload. Overview This workflow automatically scrapes competitor pricing pages (like ClickUp) and compares current pricing with previously stored data. It uses Bright Data to access competitor websites without being blocked and AI to intelligently extract pricing information, updating your tracking spreadsheet only when changes are detected. Tools Used n8n: The automation platform that orchestrates the workflow Bright Data: For scraping competitor websites without being blocked OpenAI: AI agent for intelligent pricing data extraction and parsing Google Sheets: For storing and comparing historical pricing data How to Install 1. Import the Workflow: Download the .json file and import it into your n8n instance 2. Configure Bright Data: Add your Bright Data credentials to the MCP Client node 3. Set Up OpenAI: Configure your OpenAI API credentials 4. Configure Google Sheets: Connect your Google Sheets account and set up your pricing tracking spreadsheet 5. Customize: Set your competitor URLs and pricing monitoring schedule Use Cases Product Teams: Monitor competitor feature and pricing changes for strategic planning Sales Teams: Stay informed of competitor pricing to adjust sales strategies Marketing Teams: Track competitor messaging and positioning changes Business Intelligence: Build comprehensive competitor analysis databases Connect with Me Website: https://www.nofluff.online YouTube: https://www.youtube.com/@YaronBeen/videos LinkedIn: https://www.linkedin.com/in/yaronbeen/ Get Bright Data: https://get.brightdata.com/1tndi4600b25 (Using this link supports my free workflows with a small commission) #n8n #automation #competitoranalysis #pricingmonitoring #brightdata #webscraping #competitortracking #marketintelligence #n8nworkflow #workflow #nocode #pricetracking #businessintelligence #competitiveanalysis #marketresearch #competitormonitoring #pricingdata #websitemonitoring #competitorpricing #marketanalysis #competitorwatch #pricingalerts #businessautomation #competitorinsights #markettrends #pricingchanges #competitorupdates #strategicanalysis #marketposition #competitiveintelligence
by Yaron BeenThis workflow pulls news articles from NewsAPI, Mediastack, and CurrentsAPI on a scheduled basis. Each provider’s results are normalized into a consistent schema, then written into your database (NocoDB by default). Use case: automated aggregation of categorized news for content pipelines, research agents, or editorial queues. What You Must Update Before Running 1. API Keys Replace all placeholder keys: call newsapi.org - Top Headlines → update API_KEY in URL call newsapi.org - categories → update API_KEY call mediastack → update "ACCESS_KEY" in JSON call currentsapi → update "API_KEY" param 2. Database Connection Workflow uses NocoDB to store results. You must: Update the NocoDB API Token credential to your own Ensure your table includes the fields used in the create operations (source_category, title, summary, author, sources, content, images, publisher_date, etc.) If you prefer Google Sheets, Airtable, or another DB: Replace each NocoDB node with your equivalent “create row” operation The Set nodes already provide all normalized fields you need 3. Scheduling All schedulers are disabled by default. Enable the following so the workflow runs automatically: NewsAPI – Top Headlines NewsAPI – Categories Mediastack CurrentsAPI You may change the run times, but all four must be scheduled for the workflow to function as designed. What You Can Configure 1. Categories Defined in: newsapi.org categories mediastack categories Edit these arrays to pull only the categories you care about or to match your API plan limits. 2. Article Limits Adjust article_limit in: newsapi.org categories mediastack categories currentsapi config
by Joel GambleTurn any prompt into structured web data. Send a POST request with a natural language prompt and an optional JSON schema, and get back clean, structured results scraped from the web by an AI agent powered by Firecrawl. Use Cases Data Enrichment: Feed company names or URLs from your CRM and get back structured firmographic data (industry, funding, team size, tech stack). Lead Generation: Ask the agent to find pricing, contact pages, or product details for a list of competitors. Market Research: Extract structured pricing plans, feature comparisons, or product catalogs from any website. Content Aggregation: Pull structured news, events, or job postings from across the web on a schedule. Sales Intelligence: Enrich prospect lists with company info, recent news, or tech stack details before outreach. How It Works `` POST /webhook/scrape-agent ` 1. Receive Scrape Request receives a POST request with prompt and an optional output_schema. 2. Validate Output Schema checks the schema. If none is provided, it falls back to a permissive default. If the schema is malformed, it returns a clear error via Return Schema Error. 3. Research & Extract Web Data takes the prompt and uses the full Firecrawl toolkit to research the web: Search (/search): Finds relevant pages and sources across the web. Scrape (/scrape): Extracts clean, structured content from any URL. Interact (interactContext, interact, interactStop): Lets the agent interact with scraped pages in a live session. After scraping a page, the agent can click buttons, fill forms, navigate dynamic content, and extract data that static scraping cannot reach, all without managing sessions manually. This combination gives the AI agent complete web navigation capabilities. It can discover sources, read pages, and interact with dynamic content autonomously. 4. Format Response to Schema (Structured Output Parser) formats the agent's response to match the provided (or default) schema. 5. Return Structured Results sends the structured JSON back to the caller. Setup Requirements Firecrawl API Key: Sign up at firecrawl.dev and grab your API key. Connect it in the Firecrawl credential nodes. LLM Provider: Configure your Primary Chat Model and Fallback Chat Model nodes (e.g., OpenRouter, OpenAI, Anthropic). The template uses two model nodes for reliability, plus a separate Parser Chat Model for the output parser. n8n Instance: Self-hosted or cloud. Make sure the webhook node is set to accept POST requests. API Reference Endpoint ` POST https://your-n8n-instance/webhook/scrape-agent ` Request Body | Field | Type | Required | Description | |-------|------|----------|-------------| | prompt | string | Yes | Natural language instruction for the agent | | output_schema | object | No | JSON Schema defining the desired output structure | Response Returns a JSON object matching the provided schema, or a flexible object if no schema was given. --- Testing Examples 1. Basic Request (No Schema) The agent decides the output structure on its own. `bash curl -X POST "https://your-n8n-instance/webhook/scrape-agent" \ -H "Content-Type: application/json" \ -d '{ "prompt": "Find the latest pricing for Firecrawl" }' | jq ` Expected output: A JSON object with whatever structure the agent finds most appropriate for the data. Since no schema was provided, the internal default ({ "type": "object", "additionalProperties": true }) is used. 2. Request With a Custom Schema You define exactly the shape of data you want back. `bash curl -X POST "https://your-n8n-instance/webhook/scrape-agent" \ -H "Content-Type: application/json" \ -d '{ "prompt": "Find the latest pricing for Firecrawl", "output_schema": { "type": "object", "properties": { "source": { "type": "string" }, "plans": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string" }, "price": { "type": "string" }, "credits": { "type": "string" }, "highlights": { "type": "array", "items": { "type": "string" } } } } } } } }' | jq ` Expected output: `json { "output": { "source": "https://www.firecrawl.dev/pricing", "plans": [ { "name": "Free", "price": "$0 (one-time)", "credits": "500 credits (one-time)", "highlights": [ "Scrape up to 500 pages", "2 concurrent requests", "Low rate limits", "No credit card required" ] }, { "name": "Hobby", "price": "$16/month (billed yearly, save $38)", "credits": "3,000 credits / month", "highlights": [ "Scrape up to 3,000 pages", "5 concurrent requests", "Basic support", "$9 per extra 1k credits" ] } ] } } ` 3. Invalid Schema (String Instead of Object) `bash curl -X POST "https://your-n8n-instance/webhook/scrape-agent" \ -H "Content-Type: application/json" \ -d '{ "prompt": "Find the latest pricing for Firecrawl", "output_schema": "not a valid schema" }' | jq ` Expected output: `json { "error": true, "message": "Invalid output_schema: must be a JSON object with a valid 'type' property (object, array, string, number, boolean)", "example_schema": { "type": "object", "properties": { "name": { "type": "string" }, "price": { "type": "number" } } } } ` 4. Invalid Schema (Array Instead of Object) `bash curl -X POST "https://your-n8n-instance/webhook/scrape-agent" \ -H "Content-Type: application/json" \ -d '{ "prompt": "Find the latest pricing for Firecrawl", "output_schema": [1, 2, 3] }' | jq ` Expected output: Same error response as above. 5. Invalid Schema (Missing type Property) `bash curl -X POST "https://your-n8n-instance/webhook/scrape-agent" \ -H "Content-Type: application/json" \ -d '{ "prompt": "Find the latest pricing for Firecrawl", "output_schema": { "properties": { "name": { "type": "string" } } } }' | jq ` Expected output: Same error response as above. 6. Invalid Schema (Invalid type Value) `bash curl -X POST "https://your-n8n-instance/webhook/scrape-agent" \ -H "Content-Type: application/json" \ -d '{ "prompt": "Find the latest pricing for Firecrawl", "output_schema": { "type": "banana" } }' | jq ` Expected output: Same error response as above. --- Workflow Architecture ` Receive Scrape Request (POST) | v Validate Output Schema |--- Error --> Return Schema Error (error JSON) |--- Success --> Research & Extract Web Data (AI Agent) | |--- Primary Chat Model |--- Fallback Chat Model |--- Search & Scrape: | - /search with Firecrawl | - /scrape with Firecrawl |--- Interact Tool: | - Interact context with Firecrawl | - Execute interaction with Firecrawl | - Stop interaction with Firecrawl | v Return Structured Results | |--- Format Response to Schema (Output Parser) | |--- Parser Chat Model ` Schema Validation Logic The Validate Output Schema node runs this validation before passing data to the agent: If output_schema is missing or null, the default permissive schema is used: { "type": "object", "additionalProperties": true }. If output_schema is present, it must be a JSON object (not a string, array, or primitive). It must have a type property with a valid value: object, array, string, number, or boolean. If validation fails, the workflow returns an error response with a helpful message and example schema. Notes The Format Response to Schema node (Structured Output Parser) requires the schema to be passed as a JSON string. The expression {{ JSON.stringify($('Validate Output Schema').item.json.output_schema) }}` handles this conversion. The agent has access to Firecrawl's full toolkit: search, scrape, and interact. With all three connected, the agent has complete web navigation powers. It can discover sources via search, extract content via scrape, and interact with dynamic JavaScript-heavy pages via interact. The interact tools let the agent scrape a page first and then continue working with it in a live session, clicking buttons, filling forms, and navigating deeper, all without manual session management. The agent autonomously decides which tools to use based on the prompt. Response times vary depending on the complexity of the prompt and how many pages the agent needs to visit. Simple lookups take a few seconds; deep research can take longer.
by FirecrawlThis n8n template automates the collection, storage, and summarization of technology news and discussions from the n8n Community and Reddit (r/n8n). If you like staying informed but want to reduce daily manual browsing distractions, this workflow is perfect for you. How it works User Intent Detection: The workflow receives your Telegram message and uses an AI Agent (powered by Gemini/Groq) to intelligently classify your intent (Search, Deep-dive, or Casual Chat). Multi-Platform Scraping: Automatically fetches the latest discussions, topics, and comments from the official n8n Community and Reddit via HTTP Requests. Smart Summarization: Extracts complex HTML content and uses the AI to provide concise overviews, or deep-dives into specific forum threads with full contextual history. Seamless Delivery: Formats the AI's response into clean, chunked Telegram messages (handling limits smoothly) and delivers it straight to your chat. Daily Pulse: A scheduled trigger automatically aggregates the hottest topics every morning to keep you updated. How to use 1. Connect your Telegram Bot account to n8n. 2. Connect your Google Gemini and Groq API credentials. 3. (Optional) Connect your MongoDB instance if you want to enable long-term persistent chat memory. 4. Update the Chat ID in the Telegram nodes to route the scheduled Daily Pulse directly to your personal account or group. 5. Activate the workflow and start messaging your new AI assistant. Requirements n8n Version: Built and tested on n8n 2.9.4+. (Note: You may encounter errors on older versions. It is highly recommended to update to the latest n8n version to use this workflow effectively). Telegram Bot token. Google Gemini & Groq API keys. (Optional) MongoDB connection string. Customizing this workflow Change the AI Model: Swap the Gemini or Groq nodes for OpenAI, Claude, or any other supported LLM. Expand the Sources: Duplicate the HTTP Request nodes and adjust the HTML extractors to monitor other platforms like Discord or StackOverflow. Adjust the Schedule: Prefer chat over daily emails? Change the Cron expression to receive weekly summaries or disable it entirely to rely purely on on-demand Telegram queries. --- About the Author Created by: Nguyen Thieu Toan (Jay Nguyen) Email: me@nguyenthieutoan.com Website: nguyenthieutoan.com Company: CEO/Founder of GenStaff (genstaff.net) - AI Staffing Solutions Socials (Facebook / X / LinkedIn): @nguyenthieutoan More templates: n8n.io/creators/nguyenthieutoan
by Nguyễn Thiệu Toàn (Jay Nguyen)