Grab content from another website daily

Question

Here is my problem. I am creating a website which has a "news" tab. What i want on the news tab is updated content from another news website.

Is there any way to grab plain text posted on another website, post it on a news tab in my website, and update automatically when the website posts new content? Can anybody push me in the right direction so i can learn how to do this?

I know HTML very well, but lack skill in PHP and Javascript. What do i have to learn in order to pull this off?

http://stackoverflow.com/questions/3382134/how-to-grab-content-from-another-website — Edward Ruchevits, Jan 17 '13 at 21:47
Alternatively, the other site may have (probably has) an RSS feed for their news, which you could look at grabbing the content from. Probably far easier than scraping. — newtron, Jan 17 '13 at 21:48
I would think you'd have to poll it. Like check every 2 minutes or so, and see if the latest news matches your stored latest news. If not, add it. Unless you subscribe to RSS, which I don't know how to do programatically, you couldn't get push notifications. I wouldn't be surprised if there's options I don't know of though :) — Ian, Jan 17 '13 at 21:48
It would be cool if there was a really simple way to syndicate content like that. — MikeSmithDev, Jan 17 '13 at 21:55
You could use [this tool](http://codecanyon.net/item/javascript-web-scraper/8598806) to easily scrap html. — Faraz Kelhini, Aug 20 '14 at 19:21

score 2 · Answer 1 · answered Jan 17 '13 at 22:14

Look up Curl... it is in php. http://php.net/manual/en/book.curl.php

Here is a nice video on it, that might be related to something you're trying to pull off. http://www.youtube.com/watch?v=PvEJz6du7R0

Here is also some code, to get the source code of a website using curl.

<?php

$ch = curl_init("http://www.example-webpage.com/file.html");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
$content = curl_exec($ch);
curl_close($ch);
echo $content;

?>

One more way of doing what you want, is to use an iframe within a div...

<html>
<head>
<title></title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

<style type="text/css">
<!--
#container{
    width:300px;
    height:100px;
    border:1px solid #000; 
    overflow:hidden;
    margin-left:50%;
    margin-top:5%;

}
#container iframe {
    width:1000px;
    height:750px;
    margin-left:-734px;
    margin-top:-181px;   
    border:0 solid;
 }
-->
</style>

</head>
<body>

<div id="container">
<iframe src="http://www.w3schools.com/" scrolling="no"></iframe>
</div>

</body>
</html>

Some websites don't allow you to iframe their site, so this might not work. Example, you can't iframe google, youtube, yahoo, and others.

Hope this helped :D

Oh wow, great! That curl grabs the source code? How can i filter the source code to only be the part of the code i want? I havent watched the video yet, but i will when i actually get to a computer, im doing this question on the go lol. — Brett Merrifield, Jan 17 '13 at 22:18
@BrettMerrifield try the html way first, so you don't have to go through php if you don't know it.... Just copy my html code above, and make miner tweaks accordingly to the margins... It is showing "http://w3schools.com/" website right now.... What is the website you are trying to get content from? — Brian Smith, Jan 17 '13 at 22:29

score 1 · Answer 2 · answered Jan 17 '13 at 21:51

1

You'll need to use file_get_contents and parse the html for what you want. If you want it to update periodically, you'll want to run this script on a "cron task".

If the news site has an RSS feed you could parse that instead, more effectively using SimpleXML

answered Jan 17 '13 at 21:51

Emery King

3,550
23
34

I don't understand, im very uneducated with anything other than html/css. – Brett Merrifield Jan 17 '13 at 21:54
1

@BrettMerrifield Then perhaps you should try to learn the basics of PHP before attempting something like this... – rink.attendant.6 Jan 17 '13 at 21:57
Oh okay, how can i check for an rss feed? Of it has one there should be a button like RSS on the news site, right? – Brett Merrifield Jan 17 '13 at 21:58

score 1 · Accepted Answer · answered Jan 17 '13 at 22:01

This book has a section that demonstrates reading of data from another website and parsing it using PHP. Chapter 10, pg 328 "Accessing other websites".

http://www.amazon.com/PHP-Advanced-Object-Oriented-Programming-QuickPro/dp/0321832183/

Though, if you're new to PHP, and advanced Book is no place to start. I would recommend either of the following to get you started down that road.

http://www.amazon.com/PHP-MySQL-Dynamic-Web-Sites/dp/0321784073/

or

http://www.amazon.com/PHP-Web-Visual-QuickStart-Guide/dp/0321733452/

You may be able to cobble together what you need using the Advanced book, but the best way to use advanced skills is to start learning as a beginner!

Is PHP the best solution here? Now that i realize how complex a PHP script like this would be, it cant be done. My time margin is 2 weeks and i wont be able to pull it off. Must i go with a simple rss instead? How can i do something like this? — Brett Merrifield, Jan 17 '13 at 22:11

Grab content from another website daily

3 Answers3

Linked