Title:

Cassiopeia – Towards a Distributed and Composable Crawling Platform, Journal of Telecommunications and Information Technology, 2014, nr 2

Creator:

Siwik, Leszek ; Włodarczyk, Kamil ; Marcjan, Robert

Subject and Keywords:

SEDA ; event-driven processing ; composable software ; Web crawler ; event-driven architecture ; distributed Web crawling framework

Description:

When it comes to designing and implementing crawling systems or Internet robots, it is of the utmost importance to first address efficiency and scalability issues (from a technical and architectural point of view), due to the enormous size and unimaginable structural complexity of the World Wide Web. There are, however, a significant number of users for whom flexibility and ease of execution are as important as efficiency. Running, defining, and composing Internet robots and crawlers according to dynamically-changing requirements and use-cases in the easiest possible way (e.g. in a graphical, drag & drop manner) is necessary especially for criminal analysts. The goal of this paper is to present the idea, design, crucial architectural elements, Proof- of-Concept (PoC) implementation, and preliminary experimental assessment of Cassiopeia framework, i.e. an all-in-one studio addressing both of the above-mentioned aspects

Publisher:

Instytut Łączności - Państwowy Instytut Badawczy, Warszawa

Date:

2014, nr 2

Resource Type:

artykuł

Format:

application/pdf

Resource Identifier:

ISSN 1509-4553, on-line: ISSN 1899-8852

Source:

Journal of Telecommunications and Information Technology

Language:

ang

Rights Management:

Biblioteka Naukowa Instytutu Łączności