154 17 667 5 2119 document 667 4chan-data-april-to-june.json text/plain 29efec7332d447bcd25ec7ee22791bbf MD5 3927843935 2021-07-16 22:10:43 https://researchdata.bbk.ac.uk/id/eprint/154/1/4chan-data-april-to-june.json 154 1 1 text/plain text 4chan dataset from 1st of April 2021 to 1st of July 2021 en public odc_by

4chan-data-april-to-june.json

data JSON archive 102 disk0/00/00/01/54 2021-07-29 16:05:41 2026-04-14 11:12:20 2021-07-29 16:05:41 data_collection show Prifti Ylli CMS csis public 4chan, live board, threads, posts We monitored the live /pol board on 4chan and scraped data from each thread multiple times during their time in the board and additionally (but not included in this dataset), scraped the archive status of the thread as exposed by the 4chan API and 4pleb (internet archiving service) API. Included in this dataset there is a three month extract of data scraped from 4chan during the period from 1st of April 2021 to 1st of July 2021 2021-07-16 Birkbeck College, University of London 10.18743/DATA.00154 no_funder yes 2031-07-29 bida metadata_and_data_files We used a distributed scraping system to collect this information and pointed a combination of 14 nodes, 3 clusters and 100 running instances per minute to scrap the data at high frequency, due to the ephemeral characteristics of 4chan. The scraping system was configure as a breath-first scraping system and MongoDB was used as document storage configured as a replica-set made of one primary, three secondaries and one arbiter. Extracted as JSON from MongoDB Source This is a collection of publicly available, anonymous at source and otherwise ephemeral data. 2021-04-01 2021-07-01 2021-04-01 2021-07-01