Python

Search

how to know google index of a page using python

import requests
import pandas as pd
import time
from bs4 import BeautifulSoup
from urllib.parse import urlencode

seconds = 3
proxies = {
    'https' : 'https://localhost:8123',
    'http' : 'http://localhost:8123'
    }

user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36'
headers = { 'User-Agent' : user_agent}

df = pd.read_excel('url_links.xlsx')
for i in range(0, len(df)):
    line = df.loc[i,'links']
    #print(line)
    if line:
        query = {'q': 'site:' + line}
        google = "https://www.google.com/search?" + urlencode(query)
        data = requests.get(google, headers=headers)
        data.encoding = 'ISO-8859-1'
        soup = BeautifulSoup(str(data.content), "html.parser")
        try:
            check = soup.find(id="rso").find("div").find("div").find("div").find("div").find("div").find("a")["href"]
            print("URL is Index ")
        except AttributeError:
            print("URL Not Index")
        time.sleep(float(seconds))
    else:
        print("Invalid Url")

Comment

PREVIOUS	NEXT

Code Example
Python :: python getpass save file
Python :: how to add list toa key in ict
Python :: global variable not accessible withing thread
Python :: substituir valor simbólico por valor real em uma equação Python
Python :: obtenir coordonnees souris python
Python :: Python Root finding code
Python :: placeholder in model form
Python :: rounding with .2g gives strange results
Python :: tkinter file dialog multiple file types
Python :: voilion plot
Python :: 2checkout ipn validation response python
Python :: python how to dump exception stak
Python :: python generator in while loop
Python :: how to kick and ban members with discord.py
Python :: how can space be bent
Python :: disable json dumps encode
Python :: convert to lowercase command python
Python :: check internet speed using python
Python :: execute command dynamo civid
Python :: py draw matrix of black square and white circle
Python :: assertionerror and buffererror are derived from which built-in python class
Python :: python get all keys in dict having value in range
Python :: smallest string with a given numeric value
Python :: matplotlib add abline
Python :: Display complete information about the DataFrame
Python :: dataset to list python
Python :: How to subtract all the numbers in a list from the first number?
Python :: python denest list of anything
Python :: python backtest
Python :: dynamic list in python

Search

PYTHON

how to know google index of a page using python

ADD CONTENT