Getting blank return when using GoQuery to get video src

Question

I'm trying to get the .mp4 video source of a vine, using GoQuery. However when I run it, I get nothing, no error, or return. Just a blank line.

package main

import (
  "fmt"
  "log"

  "github.com/PuerkitoBio/goquery"
)

func getMP4URL() {
  doc, err := goquery.NewDocument("https://vine.co/v/MlWtKgwh7WY")
  if err != nil {
    log.Fatal(err)
  }

  doc.Find(".vine-video-container").Each(func(i int, s *goquery.Selection) {
    mp4, _ := s.Find("video").Attr("src")
    fmt.Printf("MP4: %s", mp4)
  })
}

func main() {
  getMP4URL()
}

Is this a problem with my code, or with vine itself?

Use your browser inspection tool to compare the DOM state with the raw HTML source. You will probably see that the video container is present in the DOM but not in the HTML source. Probably due to browser detection code or something similar. — nemo, Aug 27 '14 at 03:18

score 0 · Answer 1 · answered Aug 27 '14 at 02:17

0

seems that vine adds that id with javascript

if I add

html, err := doc.Html()
if err != nil {
    log.Fatal(err)
}

log.Println(html)

before doc.Find there is no .vine-video-container in the html output

answered Aug 27 '14 at 02:17

fabrizioM

46,639
15
102
119

score 0 · Answer 2 · answered Aug 27 '14 at 02:37

try this code :)

package main

import (
    "fmt"
    "log"

    "github.com/PuerkitoBio/goquery"
)

func getMP4URL() {
    doc, err := goquery.NewDocument("https://vine.co/v/MlWtKgwh7WY")
    if err != nil {
        log.Fatal(err)
    }

    doc.Find("meta").Each(func(i int, s *goquery.Selection) {
        op, _ := s.Attr("itemprop")
        if op == "contentURL" {
            fmt.Println(s.Attr("content"))
        }
    })
}

func main() {
    getMP4URL()
}

cause the mp4 url defined in the "meta" tag not the "vine-video-container" — JessonChan, Aug 27 '14 at 09:53

score 0 · Answer 3 · answered Apr 18 '16 at 16:17

Vine embed the metadata of the video in JSON format in the <script type="application/ld+json">. So you need to extract the JSON blob from the tag and decode the JSON to get the src of the video.

The following is complete working code to get the src URL of Vine video:

package main

import (
    "encoding/json"
    "github.com/PuerkitoBio/goquery"
)

type SharedContent struct {
    ContentUrl string `json:"contentUrl"`
}

type VineVideoMetadata struct {
    SC SharedContent `json:"sharedContent"`
}

func DecodeVineJsonBlob(blob string) VineVideoMetadata {
    meta := VineVideoMetadata{}
    err := json.Unmarshal([]byte(blob), &meta)
    if err != nil {
        panic(err)
    }
    return meta
}

func GetVineVideoJsonBlob(url string) string {
    doc, err := goquery.NewDocument(url)
    if err != nil {
        panic(err)
    }

    return doc.Find("script[type=\"application/ld+json\"]").Text()
}

func GetVineVideoSrc(url string) string {
    jsonBlob := GetVineVideoJsonBlob(url)
    meta := DecodeVineJsonBlob(jsonBlob)
    return meta.SC.ContentUrl
}

func main() {
    println(GetVineVideoSrc("https://vine.co/v/MlWtKgwh7WY"))
}

output:

https://mtc.cdn.vine.co/r/videos/67FAC9DFA21115619347885645824_22a564aec15.5.0.17428816123715427422.mp4?versionId=4zcm5ySoFhqUQBXU7Ehm3YOuOSjFbkg3

Getting blank return when using GoQuery to get video src

3 Answers3