n8n

n8n is an open-source workflow automation tool that allows users to create complex workflows by connecting various apps and services without extensive coding knowledge.

Key features:

• Visual workflow builder
• 400+ integrations
• Customizable with code nodes
• Self-hosting option for privacy

Puppeteer

Puppeteer is a Node.js library that provides a high-level API to control Chrome or Firefox browsers programmatically.

Key features:

• Headless browser automation
• Screenshot and PDF generation
• Web scraping capabilities
• UI testing and performance analysis

Use Cases for n8n with Puppeteer

1. Automated Web Scraping: Extract data from websites regularly.
2. Website Monitoring: Check for changes on specific web pages.
3. Automated Testing: Perform UI tests on web applications.
4. Lead Generation: Collect contact information from multiple websites.
5. Content Aggregation: Compile content from various web sources.
6. E-commerce Monitoring: Track product availability and pricing changes.

By combining n8n's workflow automation with Puppeteer's web automation capabilities, users can create sophisticated solutions for a wide range of web-based tasks and data collection needs.

Why I Need n8n with Puppeteer

I use n8n with Puppeteer for Witbe Test Automation on our Set-Top Boxes (STBs) to automate logins for streaming services like Disney+ and AppleTV+. This setup is crucial for our testing process because:

1. Automated Authentication: Puppeteer allows me to simulate the login process for streaming services, bypassing manual input requirements.
2. 2FA Bypass: Many streaming services use Two-Factor Authentication (2FA), which typically requires manual intervention. With this setup, I can fully automate the authentication process, including 2FA steps.
3. Seamless Integration: n8n allows me to integrate this authentication process into our broader test automation workflow.
4. Increased Efficiency: By automating these login processes, we significantly reduce the time and manual effort required for testing.
5. Consistent Testing: Automated logins ensure that our tests are performed under consistent conditions, improving the reliability of our test results.
6. Scalability: This approach allows us to easily scale our testing across multiple streaming services and STB configurations.

By leveraging n8n with Puppeteer, I've created a robust, automated solution that overcomes the challenge of 2FA in our test automation process, making our testing more efficient, consistent, and comprehensive.

Installation

Prerequisites

• Cloudron server with Docker installed
• Access to your Cloudron admin interface
• Docker Hub account for pushing the Docker image
• Terminal access (Mac/Linux/WSL)
• Node.js and npm installed on your local machine

0. Installing Cloudron CLI

Before proceeding with the deployment, install the Cloudron CLI on your local machine:

1. Open a terminal or command prompt on your local machine.
2. Run the following command to install the Cloudron CLI globally:

sudo npm install -g cloudron

On Windows, run the command prompt as an administrator and omit sudo:

npm install -g cloudron

3. Verify the installation:

cloudron --version

4. (Optional) Enable tab completion by adding this line to your shell configuration file (e.g., ~/.bashrc or ~/.zshrc):

. <(cloudron completion)

Then restart your terminal or run source ~/.bashrc (or the appropriate file for your shell).

5. Log in to your Cloudron server:

cloudron login my.example.com

Replace my.example.com with your Cloudron server's domain.

1. Project Setup

Clone the Cloudron n8n app repository:

git clone https://git.cloudron.io/cloudron/n8n-app.git
cd n8n-app

(Optional) If you want to work on a separate branch:

git checkout -b my-custom-n8n

2. File Structure

Ensure you have the following files in your project directory:

• Dockerfile
• start.sh
• CloudronManifest.json
• pkg/env.sh.template

3. env.sh.template

Update pkg/env.sh.template with the following content:

export EXECUTIONS_DATA_SAVE_MANUAL_EXECUTIONS=true
export EXECUTIONS_DATA_SAVE_ON_ERROR=all
export EXECUTIONS_DATA_SAVE_ON_SUCCESS=all
export N8N_LOG_LEVEL=info
export EXECUTIONS_DATA_PRUNE=true
export EXECUTIONS_DATA_MAX_AGE=672
export NODE_FUNCTION_ALLOW_EXTERNAL=ajv,ajv-formats,puppeteer

4. Dockerfile

Update the Dockerfile with the following content:

FROM cloudron/base:4.2.0@sha256:46da2fffb36353ef714f97ae8e962bd2c212ca091108d768ba473078319a47f4

RUN apt-get update && apt-get install -y --no-install-recommends \
    libatk1.0-0 \
    libatk-bridge2.0-0 \
    libcups2 \
    libdrm2 \
    libxkbcommon0 \
    libxcomposite1 \
    libxdamage1 \
    libxrandr2 \
    libasound2 \
    libpangocairo-1.0-0 \
    libpango-1.0-0 \
    libgbm1 \
    libnss3 \
    libxshmfence1 \
    ca-certificates \
    fonts-liberation \
    libappindicator3-1 \
    libgtk-3-0 \
    wget \
    xdg-utils \
    lsb-release \
    fonts-noto-color-emoji && rm -rf /var/lib/apt/lists/*

RUN wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | apt-key add - && \
    sh -c 'echo "deb [arch=amd64] https://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list' && \
    apt-get update && apt-get install -y google-chrome-stable

RUN curl -sL https://deb.nodesource.com/setup_18.x | bash - && apt-get install -y nodejs

WORKDIR /app/code
RUN npm install n8n@latest puppeteer
ENV PATH="/app/code/node_modules/.bin:$PATH"
ENV PUPPETEER_EXECUTABLE_PATH="/usr/bin/google-chrome"

COPY start.sh /app/code/
COPY pkg /app/pkg
RUN chmod +x /app/code/start.sh
EXPOSE 5678
CMD ["/app/code/start.sh"]

5. start.sh

Update start.sh with the following content:

#!/bin/bash

set -eu

mkdir -p /app/data/user/.n8n /app/data/configs /run/cloudron.cache
export VUE_APP_URL_BASE_API="${CLOUDRON_APP_ORIGIN}/"
export WEBHOOK_URL="${CLOUDRON_APP_ORIGIN}/"
export N8N_VERSION_NOTIFICATIONS_ENABLED=false
export N8N_DIAGNOSTICS_ENABLED=false
export PATH="/app/code/node_modules/.bin:$PATH"

if [[ ! -f "/app/data/env.sh" ]]; then
    cp /app/pkg/env.sh.template /app/data/env.sh
fi

source /app/data/env.sh
exec gosu cloudron:cloudron n8n start

6. CloudronManifest.json

Update CloudronManifest.json with the following content:

{
  "id": "io.n8n.cloudronapp",
  "title": "n8n",
  "version": "3.56.0",
  "httpPort": 5678,
  "addons": {
    "localstorage": {},
    "postgresql": {},
    "sendmail": { "supportsDisplayName": true }
  },
  "runtimeDirs": ["/app/code"],
  "memoryLimit": 536870912,
  "minBoxVersion": "7.6.0"
}

7. Build and Deploy

Log in to Docker Hub:

docker login

Build and push the Docker image:

docker buildx build --platform linux/amd64 -t your_dockerhub_username/io.n8n.cloudronapp:latest --push .

Install the app on Cloudron:

cloudron install --image docker.io/your_dockerhub_username/io.n8n.cloudronapp:latest

Provide Full URL where you want it be deployed:

8. Test Puppeteer

1. Open the n8n interface.
2. Create a new workflow.
3. Add a Function node and use the following code:

const puppeteer = require('puppeteer');

async function run() {
    const browser = await puppeteer.launch({
        headless: true,
        executablePath: '/usr/bin/google-chrome',
        args: ['--no-sandbox', '--disable-setuid-sandbox']
    });
    const page = await browser.newPage();
    await page.goto('https://example.com');
    const title = await page.title();
    await browser.close();
    return [{ title }];
}

return run();

4. Execute the workflow and check the output to ensure Puppeteer is working correctly.

9. Troubleshooting

If you encounter any issues:

1. Check the n8n logs in the Cloudron admin interface.
2. Verify that all required environment variables are set correctly.
3. Ensure that the memory limit is sufficient for n8n and Puppeteer to run.
4. If Puppeteer is not working, check that Chrome is installed correctly in the Docker image.