[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV

License

Notifications You must be signed in to change notification settings

shaikhsajid1111/facebook_page_scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

31 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Facebook Page Scraper

Maintenance PyPI license Python >=3.6.9

No registration, No need of API key, No limitation on number of requests. Import the library and Just Do It !

Prerequisites

  • Internet Connection
  • Python 3.6+
  • Chrome or Firefox browser installed on your machine


Installation:

Installing from source:

git clone https://github.com/shaikhsajid1111/facebook_page_scraper 

Inside project's directory

python3 setup.py install

Installing with pypi

pip3 install facebook-page-scraper


How to use?

#import Facebook_scraper class from facebook_page_scraper
from facebook_page_scraper import Facebook_scraper

#instantiate the Facebook_scraper class

page_name = "facebookai"
posts_count = 10
browser = "firefox"

facebook_ai = Facebook_scraper(page_name,posts_count,browser)

Parameters for Facebook_scraper(page_name,posts_count,browser) class

Parameter Name Parameter Type Description
page_name string name of the facebook page
posts_count integer number of posts to scrap, if not passed default is 10
browser string which browser to use, either chrome or firefox. if not passed,default is chrome



Done with instantiation?. Let the scraping begin!


For post's data in JSON format:

#call the scrap_to_json() method

json_data = facebook_ai.scrap_to_json()
print(json_data)

Output:

{
    "1739843239525955": {
        "name": "Facebook AI",
        "shares": 43,
        "reactions": {
            "likes": 129,
            "loves": 11,
            "wow": 8,
            "cares": 0,
            "sad": 0,
            "angry": 0,
            "haha": 0
        },
        "reaction_count": 148,
        "comments": 3,
        "content": "We’re transitioning the Visdom project to the team at FOSSASIA. Visdom is a flexible tool for creating, organizing, and sharing visualizations of live, rich data. It aims to facilitate visualization of remote data with an emphasis on supporting scientific experimentation. We’re excited to see where the team, in collaboration with the developer and user community, take the project.",
        "posted_on": "2021-01-05T17:22:54",
        "video": "https://www.facebook.com/facebookai/videos/1739843239525955",
        "image": [
            "https://scontent-bom1-2.xx.fbcdn.net/v/t1.0-0/s526x296/135871741_1739843246192621_8564947121610203331_o.png?_nc_cat=108&ccb=2&_nc_sid=da1649&_nc_ohc=Hk7peLe8e-cAX_xLejp&_nc_ht=scontent-bom1-2.xx&_nc_tp=30&oh=856a17109cbc4a6657dbb68564dfc568&oe=60291FC7"
        ],
        "post_url": "https://www.facebook.com/facebookai/posts/1739843239525955"
    }, ...

}

Output Structure for JSON format:

{
    "id": {
        "name": string,    
        "shares": integer,
        "reactions": {
            "likes": integer,
            "loves": integer,
            "wow": integer,
            "cares": integer,
            "sad": integer,
            "angry": integer,
            "haha": integer
        },
        "reaction_count": integer,
        "comments": integer,
        "content": string,
        "video" : string,
        "image" : list,
        "posted_on": datetime,  //string containing datetime in ISO 8601
        "post_url": string
    }
}



For saving post's data directly to CSV file

#call scrap_to_csv(filename,directory) method


filename = "data_file"  #file name without CSV extension,where data will be saved
directory = "E:\data" #directory where CSV file will be saved
facebook_ai.scrap_to_csv(filename,directory)

content of data_file.csv:

id,name,shares,likes,loves,wow,cares,sad,angry,haha,reactions_count,comments,content,video,image,post_url
1739843239525955,Facebook AI,43,129,11,8,0,0,0,0,148,3,"We’re transitioning the Visdom project to the team at FOSSASIA. Visdom is a flexible tool for creating, organizing, and sharing visualizations of live, rich data. It aims to facilitate visualization of remote data with an emphasis on supporting scientific experimentation. We’re excited to see where the team, in collaboration with the developer and user community, take the project.",https://www.facebook.com/facebookai/videos/1739843239525955,https://scontent-bom1-2.xx.fbcdn.net/v/t1.0-0/s526x296/135871741_1739843246192621_8564947121610203331_o.png?_nc_cat=108&ccb=2&_nc_sid=da1649&_nc_ohc=Hk7peLe8e-cAX_xLejp&_nc_ht=scontent-bom1-2.xx&_nc_tp=30&oh=856a17109cbc4a6657dbb68564dfc568&oe=60291FC7,https://www.facebook.com/facebookai/posts/1739843239525955


...



Parameters for scrap_to_csv(filename,directory) method.

Parameter Name Parameter Type Description
filename string name of the CSV file where post's data will be saved
directory string directory where CSV file have to be stored.



Keys of the outputs:

Key Type Description
id string Post Identifier(integer casted inside string)
name string Name of the page
shares integer share count of post
reactions dictionary dictionary containing reactions as keys and its count as value. Keys => ["likes","loves","wow","cares","sad","angry","haha"]
reaction_count integer total reaction count of post
comments integer comments count of post
content string content of post as text
video string URL of video present in that post
image list python's list containing URLs of all images present in the post
posted_on datetime time at which post was posted(in ISO 8601 format)
post_url string URL for that post


Privacy

This scraper only scrapes public data available to unauthenticated user and does not holds the capability to scrap anything private.



Tech

This project uses different libraries to work properly.



If you encounter anything unusual please feel free to create issue here

LICENSE

MIT