n8n templatesn8n templates

HN Who is Hiring Scrape

This workflow automatically scrapes the latest job-related posts from the Hacker News forum, filters relevant information using the Algolia Search API, and retrieves detailed content through the official Hacker News API. It utilizes the OpenAI GPT-4o-mini model to clean and structure the text, ultimately storing the organized job information in an Airtable database for easier management and future use. This process achieves automated collection and structured storage of job information, enhancing data usability and retrieval efficiency.

Workflow Diagram
HN Who is Hiring Scrape Workflow diagram

Workflow Name

HN Who is Hiring Scrape

Key Features and Highlights

This workflow automatically scrapes the latest posts from the "Ask HN: Who is hiring?" hiring thread on the Hacker News (HN) forum. It leverages the Algolia Search API to precisely filter relevant job postings and uses the official Hacker News API to retrieve detailed post and comment information. The workflow employs the OpenAI GPT-4o-mini model for text cleaning and structuring, converting the data into a standardized recruitment data format. Finally, the curated job information is stored in an Airtable database for easy management and subsequent use.

Core Problems Addressed

  • Automates the acquisition and updating of Hacker News hiring posts, eliminating the tedious manual search and filtering process.
  • Unifies heterogeneous text data by extracting key information (company, position, location, salary, etc.), enhancing data usability and search efficiency.
  • Enables structured storage of recruitment information, facilitating further analysis, display, or integration with other systems.

Use Cases

  • Automated aggregation of technical job postings for recruitment platforms or community operators.
  • Real-time access to the latest technical job openings for job seekers.
  • Integration of trending job information into tech blogs or news websites.
  • Automated tracking of industry hiring trends for HR teams or headhunters.

Main Workflow Steps

  1. Manual Trigger: Initiate the scraping task.
  2. Call Algolia Search API: Precisely query posts containing "Ask HN: Who is hiring?".
  3. Split Post List: Process each job post individually.
  4. Filter Posts from the Last 30 Days: Ensure information timeliness.
  5. Call Hacker News Official API: Retrieve detailed post content and all comments (job information).
  6. Extract and Clean Text Data: Use custom code nodes to remove HTML tags and special characters.
  7. Invoke OpenAI GPT-4o-mini Model: Convert unstructured text into a unified JSON structured format.
  8. Write to Airtable: Store structured recruitment data to support subsequent queries and display.

Systems and Services Involved

  • Hacker News Algolia Search API: For rapid location of recruitment-related posts.
  • Hacker News Official API: To obtain post and comment details.
  • OpenAI GPT-4o-mini Model: For intelligent text parsing and structuring.
  • Airtable: Data storage and management platform.

Target Users and Value

  • Technical community administrators: Automate collection and management of community job posts to improve operational efficiency.
  • Job seekers and tech talent: Quickly access high-quality, structured job information to support precise job hunting.
  • HR and recruitment teams: Monitor industry hiring trends in real time to assist talent sourcing and job posting.
  • Data analysts and product managers: Conduct trend analysis and product optimization based on structured recruitment data.

This workflow offers a one-stop automated solution from data collection, cleaning, intelligent parsing to storage, significantly reducing the threshold and time cost of organizing recruitment information. It is ideal for individuals or teams who need to continuously monitor technical hiring trends.