12

I downloaded my XML sitemap from the sitemap xml generator website. I placed my sitemap.xml on my public directory but when I tried to submit the sitemap.xml into google console i received the following error: General HTTP error: 404 not found HTTP Error: 404
So i coded
app.get('/sitemap.xml', function( req, res, next ) { res.header('Content-Type', 'text/xml'); res.render( 'sitemap' ); )};

And when i navigate to the 'website/sitemap.xml' I am getting the following error:

This page contains the following errors:

error on line 1 at column 42: Specification mandate value for attribute itemscope

Thanks for your help

Vic B-A
  • 338
  • 1
  • 3
  • 10

6 Answers6

11

Generate your sitemap.xml file using a tool like https://www.xml-sitemaps.com/

upload the sitemap.xml in your project

then add this to your .js file:

router.get('/sitemap.xml', function(req, res) {
res.sendFile('YOUR_PATH/sitemap.xml');
});

make sure you change YOUR_PATH for the actual path where your sitemap.xml file is.

Louis Chaussé
  • 167
  • 4
  • 11
5

Sitemaps do not have to be XML documents. A simple text file with URLs is all you need so something like below works fine. In the following example, fetchMyUrls() would be a function/method that asynchronously gets and returns the available URLs as an array of strings (URL strings).

async function index (req, res){
    return fetchMyUrls().then((urls) => {
      var str = '';
      for (var url of urls) {
        str = str + url + '\n';
      }
      res.type('text/plain');
      return res.send(str);
    });  
}
Ronnie Royston
  • 16,778
  • 6
  • 77
  • 91
5

For those looking for a way to create the XML dynamically on your code and don't want to use another library nor have a file stored in the public folder, you can use this:

app.get('/sitemap.xml', async function(req, res, next){
  let xml_content = [
    '<?xml version="1.0" encoding="UTF-8"?>',
    '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">',
    '  <url>',
    '    <loc>http://www.example.com/</loc>',
    '    <lastmod>2005-01-01</lastmod>',
    '  </url>',
    '</urlset>'
  ]
  res.set('Content-Type', 'text/xml')
  res.send(xml_content.join('\n'))
})
Ñhosko
  • 723
  • 2
  • 8
  • 25
  • does `next()` need to be called at the bottom of this function? Edit: And, just curious, could this approach also be used for generating and serving `/ads.txt` and `/robots.txt` content - i am getting 404 errors on these three routes in my logs even though my app isn't calling them specifically. – user1063287 Sep 20 '20 at 05:04
  • I think it depends on how you are handling 404 (or other errors), if your app is catching first the error and is not reaching the point where you make the calling of /robots.txt then it will not work. On my app, the error catching takes places at the very end. Just before app.listen(). – Ñhosko Sep 21 '20 at 08:52
  • 1
    And yes, I use this same approach for generating /ads.txt and /robots.txt. Only instead of using `res.set('Content-Type', 'text/xml')` for txt file you will need `res.type('text/plain')` – Ñhosko Sep 21 '20 at 08:55
  • 1
    the `next()` is only required if you are using a middleware (which I assume you are not using).. Here the official Doc: [Writing middleware for use in Express apps](https://expressjs.com/en/guide/writing-middleware.html) – Ñhosko Sep 21 '20 at 08:59
1

In my NodeJS express project and without installing any library I was able to add this to my routes with my preferred view engine (handlebar).

export const routes: RouteMapper[] = [
    {
        "/sitemap.xml": [
        {
            method: "get",
            handler: (req, res) =>
            res.sendFile("/src/views/sitemap.xml", { root: "." }),
        },
      ],
    },
];

Cheers!

Dharman
  • 30,962
  • 25
  • 85
  • 135
Sindri Þór
  • 2,887
  • 3
  • 26
  • 32
0

The best way is to create a script that would automatically generate a sitemap. In a lot of cases, the URLs should be dynamic based on data from the database.

Great package for creating the sitemap in Express is sitemap package:

STEP 1

Create a middleware that will generate the sitemap dynamically and then cache it for each next call to the server. We can extract logic in separate file called sitemap_generator.js for example, and we can define and export generate_sitemap middleware for it:

const { SitemapStream, streamToPromise } = require('sitemap');
const { Readable } = require('stream');
let sitemap;

const generate_sitemap = async (req, res, next) => {

  res.header('Content-Type', 'application/xml');

  if (sitemap) return res.status(200).send(sitemap); // If we have a cached entry send it

  let changefreq = 'weekly';

  try {

    let links = [
      { url: '', changefreq, priority: 1 },
      { url: 'aboutus', changefreq, priority: 0.9 },
      { url: 'blog', changefreq },
      { url: 'login', changefreq },
      { url: 'register', changefreq },
    ];

    // Additionally, you can do database query and add more dynamic URLs to the "links" array.

    const stream = new SitemapStream({ hostname: 'https://example.com', lastmodDateOnly: true })
    return streamToPromise(Readable.from(links).pipe(stream)).then((data) => {
      sitemap = data; // Cache the generated sitemap
      stream.end();
      return res.status(200).send(data.toString())
    });

  } catch (error) {
    return res.status(500).end();
  }
}

module.exports = { generate_sitemap };

STEP 2

Import generate_sitemap middleware from sitemap_generator.js in your server configuration file and mound it to the /sitemap.xml endpoint:

const { generate_sitemap } = require('./sitemap_generator');

...

app.get('/sitemap.xml', generate_sitemap);

That's it. Your sitemap should be available on /sitemap.xml endpoint now so navigate in the browser to that endpoint and check if it is there.

NeNaD
  • 18,172
  • 8
  • 47
  • 89
0

After generating a sitemap.xml file from a website - take https://www.xml-sitemaps.com/ for example - you can simply copy/paste:

app.use('/sitemap.xml', function (req, res, next) {
      res.type('text/xml')
      res.send(
  `<?xml version="1.0" encoding="UTF-8"?>
  <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
        <loc>https://yourwebsite.com/</loc>
        <lastmod>2023-03-28T00:00:00+00:00</lastmod>
        <changefreq>weekly</changefreq>
        <priority>1.0</priority>
     </url>
     <url>
        <loc>https://yourwebsite.com/page1/</loc>
        <lastmod>2023-03-28T00:00:00+00:00</lastmod>
        <changefreq>monthly</changefreq>
        <priority>0.9</priority>
     </url>
     <url>
        <loc>https://yourwebsite.com/page2/</loc>
        <lastmod>2023-03-28T00:00:00+00:00</lastmod>
        <changefreq>monthly</changefreq>
        <priority>0.7</priority>
     </url>
     <url>
        <loc>https://yourwebsite.com/page3/</loc>
        <lastmod>2023-03-28T00:00:00+00:00</lastmod>
        <changefreq>yearly</changefreq>
        <priority>0.6</priority>
     </url>
     <url>
        <loc>https://yourwebsite.com/page5/</loc>
        <lastmod>2023-03-28T00:00:00+00:00</lastmod>
        <changefreq>yearly</changefreq>
        <priority>0.5</priority>
     </url>
  </urlset>`);
  });
Doracahl
  • 336
  • 2
  • 14