Python

Search

python scrape data from aspx page

#pip install requests
#pip install lxml
#pip install bs4
import requests
from bs4 import BeautifulSoup

r_obj = requests.Session()
url = "aspx_website_url_here" # i.e. http://bisegrw.edu.pk/
r_soup = r_obj.get(url)
soup = BeautifulSoup(r_soup.content , "lxml")

hidden_inputs = soup.find_all("input",type="hidden")

data = {
hidden_inputs[0]['name']:hidden_inputs[0]['value'],
hidden_inputs[1]['name']:hidden_inputs[1]['value'],
hidden_inputs[2]['name']:hidden_inputs[2]['value'],
'other_input':'value',  # i.e. 'rno':'487431',
'submitButton':'submit'}

url_needed = "aspx_endpoint" # i.e. http://bisegrw.edu.pk/result-card-matric.html
final = r_obj.post(url_needed,verify=False,data=data)
soup1 = BeautifulSoup(final.content,"lxml")
detail_tab = soup1.find_all("table")

Comment

PREVIOUS	NEXT

Code Example
Python :: mylist = [“hello”, “bye”,”see ya”,”later”] phrase = mylist[1]
Python :: cos2x
Python :: matplotlib include first number in plotter
Python :: concat dataset
Python :: get_scholarly_instance()
Python :: how to do welcome in bubble letters in python
Python :: grouped bar chart with labels
Python :: use ipython magic in script
Python :: jhon wick
Python :: python-wordpress-xmlrpc get post id
Python :: python Write code that asks users to enter the year they were born. Print out how many years old they will turn in 2019.
Python :: print("python is good")
Python :: try finally return precedent
Python :: Improve the Request Change User-Agent
Python :: Python Using Global and Local variables in the same code
Python :: pip set mirror site
Python :: get external ip address python
Python :: keras imagenet
Python :: Get index for value_counts()
Python :: python check if array alternating
Python :: assemblyai
Python :: python get stringvar value
Python :: colab show all value
Python :: Extract columns of dataframe to make new dataframe
Python :: Customizing plot with axes object
Python :: restart kernel python
Python :: vectorized function
Python :: tuples of unique rows pandas
Python :: imoport python code
Python :: python ai for stock trading

Search

PYTHON

python scrape data from aspx page

ADD CONTENT