How to retrieve value from deeper nesting in xml file?

Question

I'm having trouble getting the <Name>Administrators</Name> value from a <q4:Member> <\q4:Member> nest. I have a file generated with the command ssh PC03 "gpresult /scope computer /x:\\VBoxSvr\Exchange\PC03.xml in which the GPO settings from ADDC are saved.

For the part of the file like <q4:Account> </q4:Account> I used the pandas and everything works fine

import pandas as pd
GPO_file_Path = '.\PC03.xml'
columns = ['Name', 'SettingNumber', 'SettingBoolean']
xpath_Account = './/q :Account'
namespaces_Account = {'q' : 'http://www.microsoft.com/GroupPolicy/Settings/Security'}
df_GPO_Account = pd.read_xml(GPO_file_Path, xpath=xpath_Account, namespaces=namespaces_Account)[columns]
GPO_Account_list = df_GPO_Account.values.tolist()
GPO_Account_dic = {}
for _ in GPO_Account_list:
    if str(_[1]) != 'nan':
        GPO_Account_dic[_[0]] = int(_[1])
    else:
        GPO_Account_dic[_[0]] = _[2]
print(GPO_Account_dic)

output for account settings:

{'MinimumPasswordLength': 14, 'PasswordComplexity': 'true'}

but when i want to use the same option for the <q4:UserRightsAssignment></q:UserRightsAssignment> i get this:

input:

columns = ['Name', 'Member']
xpath_UserRightsAssignment = './/q :UserRightsAssignment'
namespaces_UserRightsAssignment = {'q' : 'http://www.microsoft.com/GroupPolicy/Settings/Security'}
df_GPO_Account = pd.read_xml(GPO_file_Path, xpath=xpath_UserRightsAssignment, namespaces=namespaces_UserRightsAssignment)[columns]
print(df_GPO_Account)

output:

                        Name  Member
0    SeCreateGlobalPrivilege     NaN

I would like to get to nesting Member (one to several elements) so as to get all <Name></Name> out of it and save everything to a list:

{'SeCreateGlobalPrivilege': {'Name':'Administratorzy', 'Name':'USŁUGA'}}

At the beginning of playing with the file, I used xml.etree.ElementTree and thanks to that I got to the places I wanted, but the code was becoming too complicated and long. Is there any option in pandas to get deeper or is there a way to get in xml.etree.ElementTree faster?

xml.etree... code:

import xml.etree.ElementTree as ET
GPO_file_Path = '.\PC03.xml'
SettingName = ""
SettingValue = ""
GPO_tree = ET.parse(GPO_file_Path)
GPO_root = GPO_tree.getroot()
GPO_dic = {}

for rootChild in GPO_root:
    ComputerResults = '{http://www.microsoft.com/GroupPolicy/Rsop}ComputerResults'
    if rootChild.tag == ComputerResults:
        for ComputerResultsChild in rootChild:
            ExtensionData = '{http://www.microsoft.com/GroupPolicy/Rsop}ExtensionData'
            if ComputerResultsChild.tag == ExtensionData:
                for ExtensionDataChild in ComputerResultsChild:
                    Extension = '{http://www.microsoft.com/GroupPolicy/Settings}Extension'
                    if ExtensionDataChild.tag == Extension:
                        for ExtensionChild in ExtensionDataChild:
                            UserRightsAssignment = '{http://www.microsoft.com/GroupPolicy/Settings/Security}UserRightsAssignment'
                            if ExtensionChild.tag == UserRightsAssignment:
                                MemberNameValue =''
                                for UserRightsAssignmentChild in ExtensionChild:
                                    Name = '{http://www.microsoft.com/GroupPolicy/Settings/Security}Name'
                                    Member = '{http://www.microsoft.com/GroupPolicy/Settings/Security}Member'
                                    if UserRightsAssignmentChild.tag == Name:
                                        SettingName = UserRightsAssignmentChild.text
                                    if UserRightsAssignmentChild.tag == Member:
                                        for MemberChild in UserRightsAssignmentChild:
                                            MemberName = '{http://www.microsoft.com/GroupPolicy/Types}Name'
                                            if MemberChild.tag == MemberName:
                                                MemberName = MemberChild.text
                                                MemberNameValue += MemberName + ', '
                                SettingValue = MemberNameValue
                                if SettingName != "":
                                    GPO_dic[SettingName] = SettingValue
                                else:
                                    print(f'Error. Key not set')

print(GPO_dic)

output:

{'SeCreateGlobalPrivilege': 'Administratorzy, USŁUGA, USŁUGA LOKALNA, USŁUGA SIECIOWA, }

What I expect:

help with <q4:Name>SeCreateGlobalPrivilege</q4:Name> where the result will be a dictionary with data: {'SeCreateGlobalPrivilege': {'Name':'Administratorzy', 'Name':'USŁUGA', 'Name':'USŁUGA LOKALNA' , 'Name':'USŁUGA SIECIOWA'}}. I don't expect a ready-made solution, but hints on how to approach the subject.

example from my file:

<?xml version="1.0" encoding="utf-16"?>
<Rsop xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.microsoft.com/GroupPolicy/Rsop">
    <ComputerResults>
        <ExtensionData>
            <Extension xmlns:q4="http://www.microsoft.com/GroupPolicy/Settings/Security" xsi:type="q4:SecuritySettings" xmlns="http://www.microsoft.com/GroupPolicy/Settings">
                <q4:Account>
                    <GPO xmlns="http://www.microsoft.com/GroupPolicy/Settings/Base">
                        <Identifier xmlns="http://www.microsoft.com/GroupPolicy/Types">{DE1708C7-FC4D-491C-942D-72CC5693DDC5}</Identifier>
                        <Domain xmlns="http://www.microsoft.com/GroupPolicy/Types">universum.local</Domain>
                    </GPO>
                    <Precedence xmlns="http://www.microsoft.com/GroupPolicy/Settings/Base">1</Precedence>
                    <q4:Name>MinimumPasswordLength</q4:Name>
                    <q4:SettingNumber>14</q4:SettingNumber>
                    <q4:Type>Password</q4:Type>
                </q4:Account>
                <q4:UserRightsAssignment>
                    <GPO xmlns="http://www.microsoft.com/GroupPolicy/Settings/Base">
                        <Identifier xmlns="http://www.microsoft.com/GroupPolicy/Types">{E7712944-FF52-4E72-AE83-7EC5C2D8A959}</Identifier>
                        <Domain xmlns="http://www.microsoft.com/GroupPolicy/Types">universum.local</Domain>
                    </GPO>
                    <Precedence xmlns="http://www.microsoft.com/GroupPolicy/Settings/Base">1</Precedence>
                    <q4:Name>SeCreateGlobalPrivilege</q4:Name>
                    <q4:Member>
                        <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">Administratorzy</Name>
                    </q4:Member>
                    <q4:Member>
                        <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">USŁUGA</Name>
                    </q4:Member>
                    <q4:Member>
                        <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">USŁUGA LOKALNA</Name>
                    </q4:Member>
                    <q4:Member>
                        <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">USŁUGA SIECIOWA</Name>
                    </q4:Member>
                </q4:UserRightsAssignment>
            </Extension>
            <Name xmlns="http://www.microsoft.com/GroupPolicy/Settings">Security</Name>
        </ExtensionData>
    </ComputerResults>
</Rsop>

Can you edit your question and add the exact expected output from your sample xml? — Jack Fleeting, Jun 05 '23 at 10:06

Jack Fleeting · Answer 1 · 2023-06-05T16:25:14.783

I'm not entirely sure about what your are trying to do (you refer to dataframes, lists, dictionaries, etc.) so I'll just show you how to extract the relevant data from your sample xml, and you can take it from there.

First, in this case I would use lxml instead of ElementTree because of it's better xpath support.

from lxml import etree

Parse your file with lxml

you may have to fiddle with the encoding for this, but it should work.

Then:

#take care of the namespaces:
ns = {"q4": "http://www.microsoft.com/GroupPolicy/Settings/Security"}
#search for the relevant info:
members = GPO_root.xpath('//q4:Name[text()="SeCreateGlobalPrivilege"]/following-sibling::q4:Member//*',namespaces=ns)
for member in members:
    print(member.text)

The output from your sample xml should be

Administratorzy
USŁUGA
USŁUGA LOKALNA
USŁUGA SIECIOWA

You can now add each one to your desired structure.

score 0 · Answer 2 · answered Jun 05 '23 at 13:50

Perhaps xmltodict is of interest:

import xmltodict
from   pathlib import Path

namespaces = dict.fromkeys([
   'http://www.microsoft.com/GroupPolicy/Rsop', 
   'http://www.microsoft.com/GroupPolicy/Settings', 
   'http://www.microsoft.com/GroupPolicy/Settings/Base',
   'http://www.microsoft.com/GroupPolicy/Settings/Security',
   'http://www.microsoft.com/GroupPolicy/Types', 
])

xmltodict.parse(
   Path('xml/microsoft.xml').read_text(), 
   process_namespaces=True,
   namespaces=namespaces,
   xml_attribs=False
)['Rsop']['ComputerResults']['ExtensionData']['Extension']

{'Account': {'GPO': {'Identifier': '{DE1708C7-FC4D-491C-942D-72CC5693DDC5}',
   'Domain': 'universum.local'},
  'Precedence': '1',
  'Name': 'MinimumPasswordLength',
  'SettingNumber': '14',
  'Type': 'Password'},
 'UserRightsAssignment': {'GPO': {'Identifier': '{E7712944-FF52-4E72-AE83-7EC5C2D8A959}',
   'Domain': 'universum.local'},
  'Precedence': '1',
  'Name': 'SeCreateGlobalPrivilege',
  'Member': [{'Name': 'Administratorzy'},
   {'Name': 'USŁUGA'},
   {'Name': 'USŁUGA LOKALNA'},
   {'Name': 'USŁUGA SIECIOWA'}]}}

You could then use pd.json_normalize if you're trying to create a pandas dataframe.

How to retrieve value from deeper nesting in xml file?

2 Answers2