How to get text of parent tag without visiting text of its child tag?

Question

I know there are other questions but i was not able to get there explanation, I have code below please help.

I want output to make a dictionary like

dictionary
{
  '[1.1]':'this is extracted text from a parent tag',
  '[1.2]':'this is child tag text',
  '[1.3]':'this is child tag text',
  '[1.4]':'this is child tag text'
}

But the problem is that i am getting the text of parent tag plus child tag in [1.1] not only the parent tag.

I tried others solution but not able to get through it. Please help somebody in some easy way .

My code is here,

from bs4 import BeautifulSoup
import requests

headers = requests.utils.default_headers()
headers.update({
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',
})

URL = "https://patents.google.com/patent/US20120303322A1/en"

content = requests.get(URL, headers=headers)
soup = BeautifulSoup(content.text,'html.parser')

independent_claim_tag = soup.find('div',{'class':'claim'})

claimdictionary = {}

# While loop to get all the independent claims tag works perfectly!!
while(independent_claim_tag):
    base = independent_claim_tag.find("div", {"class":"claim"})['num'].lstrip('0')
    print(independent_claim_tag.prettify())
    print('-------')
    elementTags = independent_claim_tag.find_all('div', {'class':'claim-text'})
    i = 1
    for tag in elementTags:
        key = "[ "+str(base)+"."+str(i)+" ] "
        ######################
        # some code need to be here to get only parent tag text for [1.1]
        value = tag.get_text()
        ######################      
        claimdictionary[key.strip()] = value.strip()
        print("[ "+str(base)+"."+str(i)+" ] "+tag.get_text())
        i = i + 1
    print('-------')
    ##################
    ##################
    print("Number of claim Element: "+str(len(independent_claim_tag.find_all('div',{'class':'claim-text'}))))
    print("---- Next Sibling")
    independent_claim_tag = independent_claim_tag.find_next_sibling('div',{'class':'claim'})


print(claimdictionary)

HTML Tag which i need to extract

<div class="claim">
 <div class="claim" id="CLM-00001" num="00001">
  <div class="claim-text">
   <b>
    1
   </b>
   . A computer readable storage medium comprising a set of instructions which, if executed by a processor, cause a computer to:
   <div class="claim-text">
    receive data corresponding to a computing node;
   </div>
   <div class="claim-text">
    identify a processor usage, a memory usage and an input/output usage based at least in part on the data corresponding to the computing node; and
   </div>
   <div class="claim-text">
    determine a compute usage value for the computing node based at least in part on the processor usage, the memory usage and the input/output usage.
   </div>
  </div>
 </div>
</div>

Number of claim Element: 4

Here Number of claim Element 4 means

{
 '[1.1]' : '1. A computer readable storage medium comprising a set of instructions which, if executed by a processor, cause a computer to:',
'[1.2]' : 'receive data corresponding to a computing node;',
'[1.3]' : 'identify a processor usage, a memory usage and an input/output usage based at least in part on the data corresponding to the computing node; and',
'[1.4]' : 'determine a compute usage value for the computing node based at least in part on the processor usage, the memory usage and the input/output usage.'
}

Updation: This is my output after some Updation

<div class="claim">
 <div class="claim" id="CLM-00001" num="00001">
  <div class="claim-text">
   <b>
    1
   </b>
   . A computer readable storage medium comprising a set of instructions which, if executed by a processor, cause a computer to:
   <div class="claim-text">
    receive data corresponding to a computing node;
   </div>
   <div class="claim-text">
    identify a processor usage, a memory usage and an input/output usage based at least in part on the data corresponding to the computing node; and
   </div>
   <div class="claim-text">
    determine a compute usage value for the computing node based at least in part on the processor usage, the memory usage and the input/output usage.
   </div>
  </div>
 </div>
</div>

-------
[ 1.1 ]  1. A computer readable storage medium comprising a set of instructions which, if executed by a processor, cause a computer to:
receive data corresponding to a computing node; identify a processor usage, a memory usage and an input/output usage based at least in part on the data corresponding to the computing node; and determine a compute usage value for the computing node based at least in part on the processor usage, the memory usage and the input/output usage.
[ 1.2 ] receive data corresponding to a computing node;
[ 1.3 ] identify a processor usage, a memory usage and an input/output usage based at least in part on the data corresponding to the computing node; and
[ 1.4 ] determine a compute usage value for the computing node based at least in part on the processor usage, the memory usage and the input/output usage.
-------
Number of claim Element: 4
---- Next Sibling
<div class="claim">
 <div class="claim" id="CLM-00008" num="00008">
  <div class="claim-text">
   <b>
    8
   </b>
   . A system comprising:
   <div class="claim-text">
    a processor; and
   </div>
   <div class="claim-text">
    a computer readable storage medium including a set of instructions which, if executed by the processor, cause the system to,
    <div class="claim-text">
     receive data corresponding to a computing node,
    </div>
    <div class="claim-text">
     identify a processor usage, a memory usage and an input/output usage based at least in part on the data corresponding to the computing node, and
    </div>
    <div class="claim-text">
     determine a compute usage value for the computing node based at least in part on the processor usage, the memory usage and the input/output usage.
    </div>
   </div>
  </div>
 </div>
</div>

-------
[ 8.1 ]  8. A system comprising:
a processor; and a computer readable storage medium including a set of instructions which, if executed by the processor, cause the system to,
receive data corresponding to a computing node,
identify a processor usage, a memory usage and an input/output usage based at least in part on the data corresponding to the computing node, and
determine a compute usage value for the computing node based at least in part on the processor usage, the memory usage and the input/output usage.

[ 8.2 ] a processor; and
[ 8.3 ] a computer readable storage medium including a set of instructions which, if executed by the processor, cause the system to,
receive data corresponding to a computing node,
identify a processor usage, a memory usage and an input/output usage based at least in part on the data corresponding to the computing node, and
determine a compute usage value for the computing node based at least in part on the processor usage, the memory usage and the input/output usage.

[ 8.4 ] receive data corresponding to a computing node,
[ 8.5 ] identify a processor usage, a memory usage and an input/output usage based at least in part on the data corresponding to the computing node, and
[ 8.6 ] determine a compute usage value for the computing node based at least in part on the processor usage, the memory usage and the input/output usage.
-------
Number of claim Element: 6
---- Next Sibling
<div class="claim">
 <div class="claim" id="CLM-00015" num="00015">
  <div class="claim-text">
   <b>
    15
   </b>
   . A computer readable storage medium comprising a set of instructions which, if executed by a processor, cause a computer to:
   <div class="claim-text">
    collect data corresponding to a computing node, wherein the data is to be associated with a processor usage, a memory usage and an input/output usage; and
   </div>
   <div class="claim-text">
    send the data to a compute usage calculation node.
   </div>
  </div>
 </div>
</div>

-------
[ 15.1 ]  15. A computer readable storage medium comprising a set of instructions which, if executed by a processor, cause a computer to:
collect data corresponding to a computing node, wherein the data is to be associated with a processor usage, a memory usage and an input/output usage; and send the data to a compute usage calculation node.
[ 15.2 ] collect data corresponding to a computing node, wherein the data is to be associated with a processor usage, a memory usage and an input/output usage; and
[ 15.3 ] send the data to a compute usage calculation node.
-------
Number of claim Element: 3
---- Next Sibling

score 1 · Accepted Answer · answered Aug 27 '19 at 07:22

You can extract() the child element inside the parent tag when you add it to your dict:

from bs4 import BeautifulSoup
import requests

headers = requests.utils.default_headers()
headers.update({
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',
})

URL = "https://patents.google.com/patent/US20120303322A1/en"

content = requests.get(URL, headers=headers)
soup = BeautifulSoup(content.text,'html.parser')

independent_claim_tag = soup.find('div',{'class':'claim'})

claimdictionary = {}

# While loop to get all the independent claims tag works perfectly!!
while(independent_claim_tag):
    base = independent_claim_tag.find("div", {"class":"claim"})['num'].lstrip('0')
    print(independent_claim_tag.prettify())
    print('-------')
    elementTags = independent_claim_tag.find_all('div', {'class':'claim-text'})
    i = 1
    for tag in elementTags:
        key = "[ "+str(base)+"."+str(i)+" ] "
        if i == 1:
            #parent
            for subtag in tag.find_all('div',{'class':'claim-text'}):
                subtag.extract()
            value = tag.get_text()
        else:
            # child
            value = tag.get_text()
        claimdictionary[key.strip()] = value.strip()
        print("[ "+str(base)+"."+str(i)+" ] "+tag.get_text())
        i = i + 1
    print('-------')
    ##################
    # some code need to be here to process parent tag text from the child tag text
    ##################
    print("Number of claim Element: "+str(len(independent_claim_tag.find_all('div',{'class':'claim-text'}))))
    print("---- Next Sibling")
    independent_claim_tag = independent_claim_tag.find_next_sibling('div',{'class':'claim'})


print(claimdictionary)

Here you can see I check the value of i and if i is 1 I remove childs inside the tag. Then I apply the get_text() method.

EDIT:

You can remove the else part and just do this too:

if i == 1:
    #parent
    for subtag in tag.find_all('div',{'class':'claim-text'}):                               
        subtag.extract()
value = tag.get_text()

thanks @Maaz for your try let me check it then say something — Nick jones, Aug 27 '19 at 07:26
it giving me error as `------- Traceback (most recent call last): File "patentextraction.py", line 38, in value = tag.get_extract() TypeError: 'NoneType' object is not callable` — Nick jones, Aug 27 '19 at 07:33
may be during the first iteration not getting the child but childs are present when `i==1` — Nick jones, Aug 27 '19 at 07:34
@RinkuYadav I didn't have this error when I tried it, what is `get_extract` ? — Maaz, Aug 27 '19 at 07:41
it's `get_text()` BeautifulSoup function to get text of tag. — Nick jones, Aug 27 '19 at 09:36
Your stacktrace is `value = tag.get_extract()` , are you sure to test my full code? — Maaz, Aug 27 '19 at 09:37
Oh its silly mistake sorry, your answer worked thanks, i just picked the if else case — Nick jones, Aug 27 '19 at 09:44

How to get text of parent tag without visiting text of its child tag?

1 Answers1