I need to extract some text from a webpage but the webpage is dynamically built(plugin). i.e I need to include a javascript SDK
<div id="fb-root"></div>
<script async defer crossorigin="anonymous" src="https://connect.facebook.net/en_US/sdk.js#xfbml=1&version=v11.0" nonce="4HbUqy4w"></script>
and then place the code where I want the plugin to appear on my page
<div class="fb-comments" data-href="https://developers.facebook.com/docs/plugins/comments#configurator" data-width="1" data-numposts="1"></div>
so in total, I have something like
<html>
<body>
<div id="fb-root"></div>
<script async defer crossorigin="anonymous" src="https://connect.facebook.net/en_US/sdk.js#xfbml=1&version=v11.0" nonce="4HbUqy4w"></script>
<div class="fb-comments" data-href="https://developers.facebook.com/docs/plugins/comments#configurator" data-width="1" data-numposts="1"></div>
</body>
</html>
Rendering this page on a browser should automatically load in some data which I now want to scrape. Is there a way to render this HTML in python? I've tried using
from requests_html import HTML
doc = # the content above
html = HTML(html=doc)
page = html.render(keep_page=True, sleep=120)
but the page is always None
Ideally, I would like something like
html_code = #here
loaded_html_code = a_package.render(html_code) # This should render my HTML which in turn causes an Iframe to be loaded.