0

I have website URL's in a DataGridView Cell when I click on the Cell the website is loaded in Chrome. I am using VS 2019 and VB.Net only I do not have ASP.Net installed.
I have tried a bunch of different concepts from some SO post that go back to 2011
With very little success I found one function that looked workable but no results I will post that code.
My question is How do I get last modified date of website?
If using VB.Net ONLY is not workable point me to a reference for other tools needed.

Public Property DateTime_LastModified As String
Dim webPAGE As String

This code is inside a click event for the DataGridView

        ElseIf gvTxType = "View" Then
        webPAGE = row.Cells(2).Value.ToString()
        'Modified: <asp:Label ID="Label1" runat="server" Text="Label"></asp:Label>
        Process.Start(webPAGE)
        GetDateTimeLastModified(requestUriString)
        Label1.Text = DateTime_LastModified
        'Dim strPath As String = webPAGE + "javascript : alert(document.lastModified)"
        'Dim strPath As String = Request.PhysicalPath
        'Server.MapPath
        'Label1.Text = System.IO.File.GetLastWriteTime(webPAGE).ToString()
        'Label1.Text = strPath '"Modified: " + System.Web.UI.GetLastWriteTime(strPath).ToString()
        'Label1.Text = strPath + "Modified:" + System.MapPath.Request.ServerVariables.Get("SCRIPT_NAME")
        'Process.Start(webPAGE)

Here is the Edit I tried from the Answer

Public Class GetURLdate1
    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click

        Dim strURL As String
        strURL = TextBox1.Text

        Dim client = New HttpClient()
        Dim msg As New HttpRequestMessage(HttpMethod.Head, strURL)
        Dim resp = client.SendAsync(msg).Result
        Dim strLastMod As String = resp.Content.Headers.LastModified.ToString

        MsgBox("Last mod as string date" & vbCrLf & strLastMod)

    'Dim lastMod As DateTime = CDate(strLastMod)

    'MsgBox(lastMod)

End Sub

Private Sub GetURLdate1_Load(sender As Object, e As EventArgs) Handles MyBase.Load

    TextBox1.Text = "https://www.youtube.com/c/stevinmarin/videos"
    '"https://stackoverflow.com/questions/70825821/how-do-i-get-last-modified-date-of-website"

End Sub

Returns NO Value for strLastMod

Vector
  • 3,066
  • 5
  • 27
  • 54
  • 1
    Looking at the docs, pretty sure the response.LastModified date isn't exactly what you're looking for. That is for when the Date/Time of the response content was last modified and goes on to talk about Entities. Maybe https://stackoverflow.com/questions/23644436/how-to-find-when-a-web-page-was-last-updated might be able to point you in the right way but I would suspect the validity of any results be 100% up to individual servers/sites – Hursey Jan 23 '22 at 19:40
  • @Hursey Thanks for the link I agree it seems about the results being individual per the site. When I use a RSS Feed Reader it has a setting that when it scrapes the site's I have selected to view the RSS Feed Reader only adds the site when new content has been posted. These are all YouTube woodworking sites. This site explains dates for sites https://www.webnots.com/how-to-find-last-updated-date-of-a-web-page/ – Vector Jan 23 '22 at 19:48
  • See my 2nd edit - I post the code to get the last modified date. At that point, you can do whatever you feel like - start a new process, jump to that web page - use your existing code and we DO NOT CARE what you do - but the code snip I just added is how you get that last modified date. If you feel like starting some process or whatever? Sure, by all means do that - but that has ZERO to do with getting the last modified date. No need to intermix the two parts of code. As noted, this code will work the same for winforms (desktop) or code behind on a asp.net (web) application. – Albert D. Kallal Jan 24 '22 at 20:18

1 Answers1

1

Ok, so it not clear if you need a routine to go get all the URL's in the grid, and update the last updated for each site/url?

Or are you looking to ONLY update the site WHEN you click on the button to jump to the site?

It is easy to do both.

I mean, say we have this markup - drop in a gridview (I let the wizards created it).

I then blow out the datasoruce control, and then remove the data soruce ID setting from the GV

So, I have this markup:

    <div style="padding:35px">
        <asp:GridView ID="GridView1" runat="server" CssClass="table" Width="65%"
            AutoGenerateColumns="False" DataKeyNames="ID" >
            <Columns>
                <asp:BoundField DataField="Url" HeaderText="Url" ItemStyle-Width="500" />
                <asp:BoundField DataField="LastUpDated" HeaderText="Last UpDated"  />
                <asp:BoundField DataField="LastVisit" HeaderText="Last Visit"  />
                <asp:TemplateField HeaderText="View" ItemStyle-HorizontalAlign="Center" >
                    <ItemTemplate>
                        <asp:Button ID="cmdJump" runat="server" Text="View" CssClass="btn"
                            OnClick="cmdJump_Click"
                            />
                    </ItemTemplate>
                </asp:TemplateField>
            </Columns>
        </asp:GridView>
        <br />
        <asp:Button ID="cmdGetAll" runat="server" Text="Upate all Last updated" CssClass="btn" />
    </div>

Ok, and my code to load the gv is this:

  Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load

    If Not IsPostBack Then
        LoadGrid()
    End If
End Sub

Sub LoadGrid()

    Using conn = New SqlConnection(My.Settings.TEST4)

        Using cmdSQL = New SqlCommand("SELECT * FROM tblSites", conn)

            Dim rstData As New DataTable
            conn.Open()
            rstData.Load(cmdSQL.ExecuteReader)
            GridView1.DataSource = rstData
            GridView1.DataBind()

        End Using

    End Using

End Sub

And now we have this:

enter image description here

Ok, so just dropped in a plane jane button into the GV. When I click on that button, I will update the last visit - not clear if you ALSO want this to update the last update for the given url?

we can do both.

So, out simple button we dropped? Well, lets have it update the last time we visit (click) to jump to the site).

Protected Sub cmdJump_Click(sender As Object, e As EventArgs)

    Dim cmdView As Button = sender
    Dim gRow As GridViewRow = cmdView.NamingContainer
    Dim PKID As Integer = GridView1.DataKeys(gRow.RowIndex).Item("ID")

    ' udpate with last visit click
    Using conn = New SqlConnection(My.Settings.TEST4)
        Using cmdSQL = New SqlCommand("update tblSites SET LastVisit = @Visit WHERE ID = @ID", conn)

            conn.Open()
            cmdSQL.Parameters.Add("@Visit", SqlDbType.DateTime).Value = Date.Now
            cmdSQL.Parameters.Add("@ID", SqlDbType.Int).Value = PKID
            cmdSQL.ExecuteNonQuery()

        End Using
    End Using

    ' Now jump to that url
    Response.Redirect(gRow.Cells(0).Text)

End Sub

And the button to go fetch and update all last updates (every row) of the given URL's cna be this:

Protected Sub cmdGetAll_Click(sender As Object, e As EventArgs) Handles cmdGetAll.Click

    Using conn = New SqlConnection(My.Settings.TEST4)

        Using cmdSQL = New SqlCommand("SELECT * FROM tblSites", conn)

            Dim rstData As New DataTable
            conn.Open()
            rstData.Load(cmdSQL.ExecuteReader)

            For Each OneRow As DataRow In rstData.Rows

                Dim client = New HttpClient()
                Dim msg As New HttpRequestMessage(HttpMethod.Head, OneRow("Url").ToString)
                Dim resp = client.SendAsync(msg).Result
                Dim lastMod As DateTimeOffset? = resp.Content.Headers.LastModified
                OneRow("LastUpDated") = lastMod.Value.ToString
            Next

            Dim da As New SqlDataAdapter(cmdSQL)
            Dim daU As New SqlCommandBuilder(da)
            da.Update(rstData)

            ' now re-load grid
            LoadGrid()

        End Using

    End Using

End Sub

So, the above has the code to update the data for all URL's.

But, your question seems to suggest that when you click on the GV row button, you ALSO want to get/save/update the given row of the GV with the last update information from the web site url?

Ok, then, we modify our click code, to update both our last click visit, and then also get that web site last update information.

So, just change the button click row code to this then:

Protected Sub cmdJump_Click(sender As Object, e As EventArgs)

    Dim cmdView As Button = sender
    Dim gRow As GridViewRow = cmdView.NamingContainer
    Dim PKID As Integer = GridView1.DataKeys(gRow.RowIndex).Item("ID")

    Dim client = New HttpClient()
    Dim msg As New HttpRequestMessage(HttpMethod.Head, gRow.Cells(0).Text)
    Dim resp = client.SendAsync(msg).Result
    Dim lastMod As DateTime = resp.Content.Headers.LastModified.ToString

    ' udpate with last visit click, and also get laste update from web site
    Using conn = New SqlConnection(My.Settings.TEST4)

        Dim strSQL As String =
            "update tblSites SET LastVisit = @Visit,LastUpdated = @LastUpDate WHERE ID = @ID"
        Using cmdSQL = New SqlCommand(strSQL, conn)

            conn.Open()
            cmdSQL.Parameters.Add("@Visit", SqlDbType.DateTime).Value = Date.Now
            cmdSQL.Parameters.Add("@LastUpDate", SqlDbType.DateTime).Value = lastMod
            cmdSQL.Parameters.Add("@ID", SqlDbType.Int).Value = PKID
            cmdSQL.ExecuteNonQuery()

        End Using
    End Using

    ' Now jump to that url
    Response.Redirect(gRow.Cells(0).Text)

End Sub

Edit: Not using asp.net

I now see in your post that you suggest you are not using asp.net. However, the above code would work near the same if you are say using a vb.net desktop + gridview. In other words, the code should be similar, and that includes the code to get that web site date.

so, you need to use follow hyper link, but the code to get the date from the web site, and even your general form layout for desktop. The results - even the way your gird looks will thus be very similar to the above web page, the overall idea here thus applies equal well to desktop or web based. I mean, if you have visual studio installed, then you could try above - as VS does install the web bits and parts for you.

Edit #2 - code to get last modified date.

There is NOT some process start, NEVER suggested to use as such. The code you need is thus this:

    webPAGE = row.Cells(2).Value.ToString()
    Dim client = New HttpClient()
    Dim msg As New HttpRequestMessage(HttpMethod.Head,webPAGE)
    Dim resp = client.SendAsync(msg).Result
    Dim lastMod As DateTime = resp.Content.Headers.LastModified.ToString

    MsgBox("DATE " & lastMod)

Edit#3: Code as win forms

Ok, so we create a blank new windows form.

Drop in a text box, drop in a button.

We have this code:

Imports System.Net.Http

Public Class GetURLdate1
    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click

        Dim strURL As String
        strURL = TextBox1.Text

        Dim client = New HttpClient()
        Dim msg As New HttpRequestMessage(HttpMethod.Head, strURL)
        Dim resp = client.SendAsync(msg).Result
        Dim strLastMod As String = resp.Content.Headers.LastModified.ToString

        MsgBox("Last mod as string date" & vbCrLf & strLastMod)

        Dim lastMod As DateTime = strLastMod

        MsgBox(lastMod)

    End Sub

    Private Sub GetURLdate1_Load(sender As Object, e As EventArgs) Handles MyBase.Load

        TextBox1.Text =
            "https://stackoverflow.com/questions/70825821/how-do-i-get-last-modified-date-of-website"

    End Sub

So, I used THIS web page for the url.

When I run this form - click on button, I see this:

enter image description here

Albert D. Kallal
  • 42,205
  • 3
  • 34
  • 51
  • I was planning on linking to the website and if the modified date was greater than today then continue the visit else just close the site. I do not know how to close a site from VB.Net code. Still working on that idea. I appreciate the effort and time spent on your answer. Will try and test the code I assume by your edit that VS does NOT need asp.net installed ? – Vector Jan 24 '22 at 18:31
  • I don't see how a web site's modified date is going to be greater than today? Not aware that occurs? As for asp.net? Yes if you installed vs, then you have asp.net also installed. So you can run and build this as web based, or just choose a desktop project. So that grid could be desktop, but you tagged this question as web based. But the code here that gets web update date would work just fine for a desktop app or web based. Only confusing part is how a web page last modified date going to be greater than today - that's not going to occur. But asp.net not required for the code to get last date – Albert D. Kallal Jan 24 '22 at 18:45
  • I agree the logic I used for Last Modified will not work. Yes I need to store dates in the DGV When I tried to write a RSS Feed Reader I did not see ASP.Net installed ? This is a Windows Form app and I style my DGV with code and the database is not BOUND to the DGV for what it is worth Still testing your code – Vector Jan 24 '22 at 18:58
  • Asp.net simply means you using vs to write and build a web based in place of desktop. All of asp.net is installed along with vs. so if using vs to build web based software then that is asp.net. if you're using vs to build a desktop then you using winforms and not asp.net. but winforms or asp.net are both equally able to run the above code to get last modified date. So you can and will use httpclient for this, and that ability to do so has zero to do with asp.net. as for the date logic? The simple point still stands - how and why would a modified date for a web page be dated beyond today? – Albert D. Kallal Jan 24 '22 at 19:54
  • Agree using web page beyond today bad idea I will post an Edit of my testing of your code. I included your code in my click event when I click on the URL in the DGV First issue was convert String to Date so I tried just a String which returns nothing SEE the Edit – Vector Jan 24 '22 at 20:02
  • Here is the error Message=Conversion from string "" to type 'Date' is not valid. And VS wants this line Dim lastMod As DateTime = CDate(resp.Content.Headers.LastModified.ToString) Does IIS need to be set to ON See this LINK https://stackoverflow.com/questions/70674832/windows-7-could-not-create-ssl-tls-secure-channel-system-net-webexception/70674920#70674920 – Vector Jan 24 '22 at 20:26
  • 1
    What does IIS have to do with this at all? See my edit #3 - I show this working as winforms example. – Albert D. Kallal Jan 24 '22 at 20:47
  • No problem - give that simple winform and simple button code a try. In fact, comment out the line that attempts to assign the string to the datetime - that can be worked on as a 2nd step and challenge. Just use the msgbox and display the returned modified date as a string. If that works, then you can work on the next step of converting the string to a date time So one small step at a time is the best way to attack this problem. – Albert D. Kallal Jan 24 '22 at 20:57
  • Posted the error and the line of code where the error starts It is in the Question OK I used a Youtube site and got Message=Conversion from string "" to type 'Date' is not valid – Vector Jan 24 '22 at 21:40
  • 1
    [You are using HttpClient wrong](https://visualstudiomagazine.com/blogs/tool-tracker/2019/09/using-http.aspx) It's meant to be instantiated once and reused over and over again. Also, the Last Modified header is by no means a required response header. There's no way to guarantee a server is going to tell you when a resource was last modified. – Heretic Monkey Jan 24 '22 at 21:52
  • @HereticMonkey I believe you about the resource not know Last Modified My question would be the RSS Feed Reader I was playing with only shows me content if the YouTube site has posted new content. So how do they know new content was posted ? – Vector Jan 24 '22 at 22:03
  • 1
    RSS is an XML-based syndication format that provides metadata beyond what is necessarily available via plain HTTP. The syndicator could be tracking last modified independently via their own database for all I know. – Heretic Monkey Jan 24 '22 at 22:11
  • @HereticMonkey So even if I use the Http Factory I may not get the Last Modified date from the YouTube site I used this as a jumping off point to try and learn what is going on https://www.webnots.com/how-to-find-last-updated-date-of-a-web-page/ – Vector Jan 24 '22 at 22:22
  • 1
    There's no guarantee that any of those methods will result in a usable last modified value. Also, in terms of YouTube, what does "last modified" even mean? They probably generate each page dynamically; so what meaning does "last modified" hold? When I look at the page for a video, I see a date, but that's the date it was uploaded. You'd need to inspect the DOM to find the selector for that element to scrape that date. – Heretic Monkey Jan 24 '22 at 22:45