Data extraction with Scrapy-II

In this post, I will discuss on the subtle features of the Scrapy framework responsible for data extraction including building a basic spider. But first, a key points to remember, that are as follows; Preliminaries From the documentation, “Scrapy spiders can return the extracted data as Python dicts” therefore to separate the key-value pair the…

Data extraction with Scrapy-I

Disclaimer: The objective of this post is purely educational in nature. There are no monetary benefits associated. Introduction In this digital age, we are surrounded by data and a majority of it is in unstructured format. The oxford dictionary defines unstructured as “Without formal organization or structure.”. Websites are a rich source of this unstructured…

Scrapy on Windows – Setup

“Scrapy is an open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.” This article will walk you through installing Scrapy (on a windows operating system). 1.    Preliminaries First, ensure the following dependencies exist on your machine; Step 1: Python version 2.7 as scrapy only…