n8n templatesn8n templates

LinkedIn Web Scraping with Bright Data MCP Server & Google Gemini

This workflow combines advanced data collection services with AI language models to automatically scrape information from personal and company pages on LinkedIn, generating high-quality company stories or personal profiles. Users can efficiently obtain structured data, avoiding the time wasted on manual operations. It also supports saving the scraped results as local files or real-time pushing via Webhook for convenient later use. This is suitable for various scenarios such as market research, recruitment, content creation, and data analysis, significantly enhancing information processing efficiency.

Workflow Diagram
LinkedIn Web Scraping with Bright Data MCP Server & Google Gemini Workflow diagram

Workflow Name

LinkedIn Web Scraping with Bright Data MCP Server & Google Gemini

Key Features and Highlights

This workflow integrates Bright Data MCP (Market Client Platform) data collection services with Google Gemini large language model to automate data scraping from LinkedIn personal and company pages and enable intelligent content generation. It efficiently extracts web information, structures the data, and automatically generates detailed company stories or personal profiles in high-quality Markdown format. Additionally, it supports saving the data as local files for convenient future use.

Core Problems Addressed

  • Automates the extraction of publicly available personal and company profiles on LinkedIn, eliminating time-consuming and error-prone manual copy-pasting.
  • Utilizes AI models to intelligently organize raw scraped data and generate refined content, enhancing information utilization and expression quality.
  • Supports real-time push of scraping and processing results via Webhook, facilitating integration with other systems or triggering subsequent automated workflows.

Use Cases

  • Market researchers needing to quickly gather detailed information and background stories of target companies.
  • Recruitment teams automatically obtaining LinkedIn profiles of candidates to assist in screening and evaluation.
  • Content creators generating introductory articles or blog posts based on company or personal profiles.
  • Data analysts performing industry or competitor analysis by rapidly collecting and formatting bulk data.

Main Workflow Steps

  1. Manually trigger the workflow start.
  2. List all crawler tools supported by Bright Data MCP.
  3. Set the target LinkedIn personal and company page URLs.
  4. Use the Bright Data MCP client to scrape personal and company page data separately, returning results in Markdown format.
  5. Parse the JSON content of the scraping results via a code node.
  6. Extract structured company details using LangChain’s Information Extractor node.
  7. Invoke the Google Gemini model to generate a complete company story or personal introduction based on the extracted information.
  8. Merge and aggregate the scraped and generated content.
  9. Send the scraped LinkedIn company and personal information via Webhook.
  10. Encode personal and company information into binary format separately and write them to local JSON files for storage.

Involved Systems and Services

  • Bright Data MCP Server: Provides powerful web crawling and data collection capabilities.
  • Google Gemini (PaLM API): AI language model supporting natural language generation and information extraction.
  • n8n Automation Platform: Serves as the workflow foundation, enabling data flow and logic control between nodes.
  • Webhook.site: Temporary URL service for receiving and testing Webhook pushes.
  • Local File System: Saves scraping results as JSON files.

Target Users and Value

  • Data scientists, market analysts, recruitment specialists, and other professionals can significantly improve LinkedIn data collection and analysis efficiency with this workflow.
  • Automation engineers and technical teams can rapidly build intelligent information processing systems based on AI and web scraping technologies.
  • Content creators and enterprise users can enhance content production quality and speed through automatically generated company stories or personal profiles.
  • Any users requiring regular bulk scraping and intelligent processing of publicly available LinkedIn profiles to support business decisions.

By integrating leading data collection and AI technologies, this workflow comprehensively enhances the acquisition and utilization efficiency of LinkedIn information, empowering users to achieve intelligent, data-driven business operations.