Questions tagged [web-scripting]
97 questions
1
vote
1 answer
How to check if the specific text on a website changed using python script
I'm trying to write a python script to check the status's display text for a specific country (ie. Ecuador)
on this website:
https://immi.homeaffairs.gov.au/what-we-do/whm-program/status-of-country-caps.
How do I keep track on that specific text…
Yos
- 11
- 1
1
vote
1 answer
When importing HTML into Google Sheets, there are issues with the order by and select col functions as well as the header
I am using this function =QUERY(INDEX(SUBSTITUTE(IMPORTHTML (I2 & "?refresh=" & I6,I3,I4), "*", )), "SELECT Col1, Col2, Col3, Col4 ORDER BY Col1 ASC")
This is returning data with the headers in between the table.
Also, can we use offset to avoid…
kevin shah
- 11
- 2
1
vote
2 answers
Selenium Python: Invalid Selector in find_element_by_css_selector (Error)
I'm doing an automation to search a certain process on a website and collect information. When requesting data collection, it returns the following error:
"Selenium Python: Invalid Selector in find_element_by_css_selector"
this is my code:…
Avestruz
- 27
- 4
1
vote
2 answers
Extract text from dynamic Web page using R
I am working on a data prep tutorial, using data from this article: https://www.nytimes.com/interactive/2021/01/19/upshot/trump-complete-insult-list.html#
None of the text is hard-coded, everything is dynamic and I don't know where to start. I've…
Dominic Comtois
- 10,230
- 1
- 39
- 61
1
vote
1 answer
problem scraping with BeautifulSoup
I am trying to scrape the url http://www.kat.ph/search/beatles/?categories[]=music using BeautifulSoup
torrents = bs.findAll('tr',id = re.compile('torrent_*'))
torrents gets all the torrents on that page ,now every element of torrents contains a…
Bunny Rabbit
- 8,213
- 16
- 66
- 106
1
vote
2 answers
How can I be able to create a folder node on companyhome in Alfresco?
I am having a hard time looking for an answer to my question. I am trying to create a folder/space in Alfresco. But I don't have any idea doing it? Can someone help me with this? I'm using Java webscript.
All I am at is this:
package…
bluestella
- 383
- 2
- 6
- 17
1
vote
0 answers
How would you recommend I automate website data entry?
I'm looking for a simple script that I can use to automate website data entry.
Trigged when the value of a cell in a gsheet changes, the script will login to the website, navigate to the correct url, fill one or two fields to select the correct item…
Steve Venton
- 11
- 1
1
vote
1 answer
how to modify html using javascript after the page loads
I am new to js and learning how to modify html using js ,i want to modify:
To...
1
vote
0 answers
Why is my VBA program not getting all of the tag names that are equal to "path" from an website?
I am wondering if anyone would please be able to help me with this issue I am having. I am trying to make a program that will find a certain icon on a web page and then click on all occurrences of it based on the tag name that the element has. The…
michdoming
- 11
- 2
1
vote
1 answer
How to pull the vedio link name imbeded in a web page along with vedio name
I am trying to pull all the video link reference from the web page along with video names, i have tried below code.
#!/usr/bin/python3
from bs4 import BeautifulSoup
import requests
import urllib
url =…
user2023
- 452
- 5
- 22
1
vote
1 answer
Image doesn't download fully with aiohttp
I am using the following to get an image, but it's not properly downloaded and only a small top portion of image is seen. The file size is less than 256 kB. What's wrong and how could I fix it?
async with aiohttp.ClientSession() as session:
…
Anonymous Guy
- 119
- 2
- 11
1
vote
4 answers
I'm learning python and am interested in using it for web-scripting. What frameworkes are out there and do I need one?
I've been learning python for use in ArcGIS and some other non-web applications. However, now that I've taken on building a personal website I am interested in using it for web development (as it is the only scripting language I currently…
Usagi
- 2,896
- 2
- 28
- 38
1
vote
1 answer
How to scrape data from PDF file in Python
I want to scrape the data present in 7th page of this PDF and move to dataframe then CSV.
Could you please help on the same?
Learnings
- 2,780
- 9
- 35
- 55
1
vote
1 answer
Read page source before POST
I want to know if there is a way to POST parameters after reading the page source. Ex: read captcha before posting ID#
My current code:
import requests
id_number = "1"
url = "http://www.submitmyforum.com/page.php"
data = dict(id = id_number, name =…
Jeremy Claus
- 13
- 2
1
vote
2 answers
While parsing any URL DOM is not loading
$ie = new-object -com internetexplorer.application
$ie.visible=$true
$ie.navigate('https://google.co.in')
while($ie.ReadyState -ne 4)
{
Write-Host "dom is loading"
$ie.ReadyState -eq 4
$ie.ReadyState -eq 3
$ie.ReadyState -eq 2
…
aditya_rahatekar
- 23
- 6