Member-only story
How To Build an Image Crawler Without Coding
Saving an image from a webpage is straightforward, simply right-click and select “save image as.” But what if you have hundreds or even thousands of images that need to be saved? Will the same trick work?
In this article, I want to show you how to quickly build an image crawler without coding. Even if you have absolutely no tech background, you should be able to nail this within 30 minutes. No matter what reason you may need the pictures for — whether that be reblogging, reselling, or machine training — the same trick can be extended to literally any websites.
Not all images are created equal. Some images can be fetched from the webpage directly, other images are triggered only by clicking the thumbnails.
In this tutorial, I will show you how to deal with each of these scenarios via a few examples.
Ready? Let’s get started.
You will need the following tools:
* Octoparse: a coding-free visual web scraping tool
* Tab Save: a Chrome plugin to save images instantly upon providing a list of URLs
Prerequisites
It would be best if you are familiar with how Octoparse works in general. Octoparse Scraping 101 may also help if you are new to the tool.
Example 1: Fetching Images Directly From a Webpage
To demonstrate, we are going to scrape dog images from Pixabay.com. To follow along, search for dogs on Pixabay.com. This should get you to this page.
1. Click “+ Task” to start a new task under Advanced Mode. Then, input the URL of the target webpage into the text box and click “Save URL.”
You should arrive here:
2. Next, we are going to tell the bot what images to fetch.
Click on the first image. The “Action Tips” now reads “Image selected, 100 similar images found.” This is great — exactly what we need. Go on to select “Select all,” then “Extract image URL in the loop.”
