ChatBot retrieve data from SharePoint On Premise Issue

Question

I've developed a chatbot that communicates with SharePoint on Premise,

When I run the chatbot in Emulator its work.

But When I run at Web that hosted outside of SharePoint, it does not work.

Herewith my screenshot of Error On Azure, From the result of Error is starting from XMLReader and SyndicationFeed

Success in Local Emulator

Herewith my Souce Code.

 private async Task ProcessRSSAsync(ITurnContext<IMessageActivity> turnContext, LuisResult luisResult, string intent, CancellationToken cancellationToken)
        {
            var questionluis = turnContext.Activity.Text;

            await turnContext.SendActivityAsync("intent recognize" + intent);

            var intentresut = intent;
            await turnContext.SendActivityAsync("Get LUIS Entity");
            await turnContext.SendActivityAsync(string.Join("\t", luisResult.Entities.Select((entityObj) => entityObj.Entity)));
            var entityfound = string.Join("\t", luisResult.Entities.Select((entityObj) => entityObj.Entity));

            string spxurl = @"https://intra.aspac.com/sites/sg/daw/_layouts/15/srchrss.aspx?k=*%20ListId:7BC0F2C3-6366-48B8-B88A-8738BE1F9C31";

            await turnContext.SendActivityAsync("Intent: " + intent.ToString() + " Entity: " + entityfound.ToString());
            ////---------------------------------------------------------------------------------------------
            //22112019
            try
            {

                //#ES09122019
                var credentials = new NetworkCredential("email@example.com", "Pa$$w0rd", "sg.kworld.com");

                var handler = new HttpClientHandler { Credentials = credentials, UseDefaultCredentials = false };
                var client = new HttpClient(handler);

                client.BaseAddress = new Uri("https://intra.aspac.com/sites/sg/daw/");

                HttpResponseMessage resp = client.GetAsync("_layouts/15/srchrss.aspx?k=" + entityfound + "*%20ListId:7BC0F2C3-6366-48B8-B88A-8738BE1F9C31").Result;

                string respString = resp.Content.ReadAsStringAsync().Result;


                if (resp.StatusCode == HttpStatusCode.OK)
                {
                    await turnContext.SendActivityAsync("Connected");

                    //Success 06122019 .
                    try
                    {
                        string spurl = @"https://intra.aspac.com/sites/sg/daw/_layouts/15/srchrss.aspx?k=*%20ListId:7BC0F2C3-6366-48B8-B88A-8738BE1F9C31";
                        XmlSecureResolver resolver = new XmlSecureResolver(new XmlUrlResolver(), spurl);
                        resolver.Credentials = new NetworkCredential("email@example.com.sg", "Pa$$w0rd", "sg.kworld.com"); 

                        XmlReaderSettings settings = new XmlReaderSettings();
                        settings.DtdProcessing = DtdProcessing.Parse;
                        settings.ValidationType = ValidationType.DTD;

                        settings.XmlResolver = resolver;



                        XmlReader reader = XmlReader.Create(spurl, settings);

                        SyndicationFeed feed = SyndicationFeed.Load(reader);

                        reader.Close();

                        var attachments = new List<Attachment>();
                        foreach (SyndicationItem item in feed.Items)
                        {
                            //Get Title,Description,URL
                            String title = item.Title.Text;
                            String description = item.Summary.Text;
                            String link = item.Links.FirstOrDefault().Uri.ToString();

                            //Hero Card
                            var heroCard = new HeroCard(
                                title: item.Title.Text,
                                // subtitle: description,
                                buttons: new CardAction[]
                                {
                         new CardAction(ActionTypes.OpenUrl,"Learn More",value:link)
                                }
                                ).ToAttachment();
                            attachments.Add(heroCard);

                        }
                        var reply = MessageFactory.Carousel(attachments);
                        await turnContext.SendActivityAsync(reply);
                        await ProcessCosmoDBStorageLUISAsync(turnContext, questionluis, intent, entityfound, respString, cancellationToken);
                    }
                    catch (Exception ex)
                    {
                        await turnContext.SendActivityAsync(ex.ToString());
                    }

                }
            }

            catch (Exception ex)
            {
                await turnContext.SendActivityAsync("Sorry,Currently Server Under Maintenace");
                await turnContext.SendActivityAsync(ex.ToString());
            }

        }

any solution for this and suggestion?

Where is this bot actually running? On the same network as the SharePoint instance? — Hilton Giesenow, Jan 07 '20 at 17:51
@HiltonGiesenow The Bot should run at the web page and same network, and not running inside the SharePoint Page. Any better suggestion? Thank you. — Eng Soon Cheah, Jan 08 '20 at 00:04

score 2 · Accepted Answer · answered Jan 08 '20 at 13:09

2

ok, I think I finally understand this better, so hopefully can put a useful reply together. Would be much easier if we had a shared whiteboard :-)

Basically, in terms of hosting a bot on the Microsoft Bot Framework Services, you need to have a registration in Azure. However, there are two different options, and both are VERY different in terms of hosting. When you "create" the resource in Azure, and search for "Bot", you'll see two options - "Web App Bot" and "Bot Channels Registration":

"Bot Channels Registration" means JUST registering your bot in Azure, but HOSTING it elsewhere.
"Web App Bot" - INCLUDES the "Bot Channels Registration" but ALSO adds hosting using an Azure Web Application (so it's a Bot registration PLUS hosting)

From the screenshot you posted, I can see you've selected (2) above, and so your bot is running inside Azure, and therefore can't connect to your on premises resource (SharePoint).

As a result, I'd suggest one of two options:

Create an Azure Application Proxy - this is basically a small gateway so that your bot HOSTED in Azure can securely talk to your on-premises SharePoint. There is in fact a specific use case for SharePoint in particular.
Delete and re-create your Azure Bot entry to instead be just a "Bot Channels Registration", and then in the "Settings" screen you can call a bot hosted at any "https" endpoint. You can then have your bot run on the local network, but it will need a live "https" address (not -that- hard to do, but you have to involve your IT team to get a live web address, like "whatever.aspac.com", and you'll need an SSL/TLS certificate so that it can run httpS instead of just http.

Which option you choose might depend on the skills and resources on your team, as well as in the organisation. For instance, the company might have Azure Application Proxy configured already, in which case that saves a lot of work. It might have a wildcard certificate, which would make option (2) easier, etc.

Either way, I hope that helped, but feel free to ask more if anything is still unclear.

answered Jan 08 '20 at 13:09

Hilton Giesenow

9,809
2
10
24

For the Option 2, Its means my Source code(Bot Application) need to host at the server? – Eng Soon Cheah Jan 09 '20 at 00:36
Yes, it should work totally fine because then the bot will be running on the local network again, where it can reach SharePoint. A very simple way to test this is to run the bot on your local machine, and to use the free NGrok tool - it will give you a temporary "real" https address that you can use in the Bot Channel registration. Don't forget to check the url when you do this - I sometimes forgot to add "/api/" in as part of the address - it should look something like https://xyz.ngrok.io/api/messages – Hilton Giesenow Jan 09 '20 at 04:33
by the way, another separate thing - for the bot you -do- have created in Azure, just check the billing - my first bot I created also using the "web app" bot template, an a pay-as-you-go plan, and accidentally incurred a $70 bill for the month when I only used it for a few days, was a sad waste. – Hilton Giesenow Jan 09 '20 at 05:17
@EngSoonCheah Did you make any progress with this? – Hilton Giesenow Jan 10 '20 at 06:31
is that running "npx serve"?after I run the npx server: Local: http://localhost:5000 and On Your network: http:10.X.X.47:5000, I should put http:10.x.x.47:5000/api/messages ? I need to wait my IT guys Holiday come back, so that I can host my Bot API/messages to the servers. – Eng Soon Cheah Jan 13 '20 at 06:09
no, it's not the same. I assume your bot is a Node.js bot (versus a C#). If you build a C# in Visual Studio and push F5, it will run a local mini web server, and usually run on port 3978, so it would be running something like http://localhost:3978. You're now "running" the website locally. NPX is the equivalent of that for Node, so you code is now "running" in a website, but with port 5000, so it's http://localhost:5000. That's only stage 1 - the problem now is it can't be accessed via the live internet as it's only on your local machine (continued, no more space) – Hilton Giesenow Jan 13 '20 at 06:19
So, the next thing you need to do, to make it accessible over the internet, while in development, is to make a temporary "real" internet address, and ideally one that's httpS not just http. To do that, you use a "tunnelling" tool, like one called NGrok, which runs on your local machine and makes a tunnel to the outside internet with a temporary live address, like httpS://abc123.ngrok.io . You can try access that address from, say, your cellphone, over your cell connection, and you'll actually see it coming in to your computer. – Hilton Giesenow Jan 13 '20 at 06:22
you need to tell NGrok what port to use, so you'd run something like: ngrok http 5000 -host-header="localhost:5000" That tells ngrok that any call to it's special address, must actually be forwarded to you local web server running the app. THEN the next step is to take that Ngrok address, and tell the BOT FRAMEWORK to use that as the "endpoint" for your bot. When you do, don't forget that the "endpoint" it needs is the ngrok part, PLUS the rest of the address to the bot itself, so it's usually something like: https://abc123.ngrok.io/api/messages – Hilton Giesenow Jan 13 '20 at 06:24
you set that on the "Settings" screen for your bot. I can't put an image here in the comments, but see the image on my answer here to see which screen you need: https://stackoverflow.com/questions/59627264/how-to-add-image-logo-for-bot-in-ms-teams/59627486#59627486 – Hilton Giesenow Jan 13 '20 at 06:25
I had change it, Azure mention Please allow "30 minutes for changes to bot settings to be reflected in all regions. Changes to icons may take up to 24 hours." – Eng Soon Cheah Jan 13 '20 at 06:33
no, don't worry about that - that's for the icon. Changing the endpoint is usually updated in about 1 minute - it's quick – Hilton Giesenow Jan 13 '20 at 06:39
@EngSoonCheah did you make any more progress with this? – Hilton Giesenow Jan 14 '20 at 08:21
for the ngrok its does not work in company laptop. Now I waiting my IT guys deployment. – Eng Soon Cheah Jan 14 '20 at 08:50
ah ok - the IT guys need to enable it to run? – Hilton Giesenow Jan 14 '20 at 08:56
By the way, I'm just setting a new blog after many years, and I've done quite a long post on some of what I've learned working with Bots behind the scenes, it might be interesting for you to read, especially relevant to this topic. See here: https://hilton.giesenow.com/how-bot-calls-actually-work – Hilton Giesenow Jan 14 '20 at 10:13
@EngSoonCheah I see you "unaccepted" on this - what happened? I assume you ran into some issues? – Hilton Giesenow Jan 16 '20 at 05:35
I think I not be-careful click. – Eng Soon Cheah Jan 16 '20 at 05:35

score 0 · Answer 2 · answered Jan 08 '20 at 08:56

i had a similar problem using a on premise database. as you are deploying your bot externally, the bot needs resources that are available on the internet, and not contained internally. It will work fine using the bot emulator because it has access to what your machine has.

Saying that, azure has developed some actions which you can use to help this problem. If you look at application proxys, that may be able to help you out.

i think thats what you mean... anyway!

ChatBot retrieve data from SharePoint On Premise Issue

2 Answers2

Linked