xmlHttpRequest displays eurosign as questionmark

Question

I use this script (from dynamicdrive) to dynamically fill div with id:

    var bustcachevar=1 //bust potential caching of external pages after initial request? (1=yes, 0=no)
    var loadedobjects=""
    var rootdomain="http://"+window.location.hostname
    var bustcacheparameter=""

    function ajaxpage(url, containerid){
        var page_request = false
        if (window.XMLHttpRequest) // if Mozilla, Safari etc
        page_request = new XMLHttpRequest();
        else if (window.ActiveXObject){ // if IE
            try {
                page_request = new ActiveXObject("Msxml2.XMLHTTP");
                }
            catch (e){
                try{
                    page_request = new ActiveXObject("Microsoft.XMLHTTP");
                    }
                catch (e){}
            }
        }
        else
        return false
        page_request.onreadystatechange=function(){
            loadpage(page_request, containerid)
        }

        if (bustcachevar) //if bust caching of external page
        bustcacheparameter=(url.indexOf("?")!=-1)? "&"+new Date().getTime() : "?"+new Date().getTime()
        page_request.open('GET', url+bustcacheparameter, true)
        page_request.send(null)
    }

    function loadpage(page_request, containerid){
        if (page_request.readyState == 4 && (page_request.status==200 || window.location.href.indexOf("http")==-1))
        document.getElementById(containerid).innerHTML=page_request.responseText
    }

Everything works fine until I load a page with for example a euro-sign in it. Codepage's are set correctly on the page but it displays a questionmark. I don't know enough javascript to fix this problem.

Thanks in advance for any advice!

NOTE: Thanks to friend I now know that saving the file you want to load using this script in UTF-8 fixes the problem. But I can't be sure that every page I load is UTF-8 encoded so my question is:

is there a way for the script to set the right charset? Is there a way to let the script adapt to the codepage of the file you want to load?

I can't see how it can be a JavaScript issue. Most likely, you are using different encodings in both documents, the server-side AJAX script and the target HTML document (or possibly relying on default encodings). — Álvaro González, Nov 10 '11 at 18:55

Felix L · Answer 1 · 2011-11-10T18:45:48.947

0

It seems like you have an encoding problem somewhere.

I highly suggest you to use UTF-8 everywhere as it is the established standard for the web. Check that the page doing the ajax call and the dynamically loaded page are encoded in UTF-8 and sent by the server with correct headers (the headers should contain something like Content-type: text/html; charset=UTF-8).

Also it is a best practice to replace exotic characters by their html firendly code in html pages to avoid such issues. Use € for €.

edited Nov 10 '11 at 18:45

answered Nov 10 '11 at 18:40

Felix L

246
1
3

Thanks for the reply. I know about the standards and as I've noted in my question, encoding the sourcefile does fix the problem. Is there a way to let javascript set the encoding or change/adapt to the file being read? The file used to be ANSI encoded instead of UTF-8 when the problem occurred. I would like to be able to load pages in different charset's since I'm not the only one who uploads file's that can be dynamically loaded. A user might not know how to save a file as UTF-8, and they shouldn't care (since you must asume the user is dumb). – C-TZ Nov 10 '11 at 18:55
Do you suggest using UTF-8 and avoiding non-ASCII characters at the same time? – Álvaro González Nov 10 '11 at 18:56
@Álvaro G. Vicario No :) I know we don't really care about the encoding if there are no non-ASCII characters... – Felix L Nov 10 '11 at 19:13
I don't think this is too practical... User's may upload files in ANSI format using special characters in it. That's a problem using this script as it is now. – C-TZ Nov 10 '11 at 19:15

score 0 · Answer 2 · answered Nov 10 '11 at 19:38

This is my hypothesis (and I think it's been confirmed by your updates):

When you write the remote document you are loading, you just open your editor, hit the € symbol in your keyboard and save. Since you never picked any encoding, your editor used the ANSI code page. And here's the issue: the ANSI code page basically depends on where you live. In Western Europe, Win-1252 is a popular choice and encodes the euro symbol as 0x80.
When you write the target HTML doc where you want to insert it, you do exactly the same and get a Win-1252 document. However, the webserver doesn't know what the encoding is. Many times, it'll default to something like ISO-8859-1 and it happens that ISO-8859-1 does not even have an euro symbol!
JavaScript reads 0x80 and writes 0x80.
The browser finds 0x80 in an HTML document that's supposedly ISO-8859-1. In such encoding, the 0x80 is actually blank.

So you don't have to fix your JavaScript code (there's nothing fixable there, mainly because there's nothing wrong there). You need to find out what your site's encoding is and generate files that actually use such encoding (advanced editors will let you choose).

Hi thanks for the response. It happens like that exactly. Even when I specify in index.php that I want lets say ISO-8859-15, which has the euro symbol, or WIN-1252, I still get the question mark instead of the euro-sign. That means that it doesn't matter what codepage I use to setup my index.php file. It all depens on the encoding I use on the file which gets loaded using javascript. Say, for example, that someone uploads a wrong charcode based file to my server and loads it with this script, is there someway on forehand that I can check for the codepage and 'convert' it somehow? — C-TZ, Nov 12 '11 at 20:07
There isn't a 100% reliable way to detect the encoding of some arbitrary text file uploaded by a random user. Why don't you simply build an upload feature with a textarea? — Álvaro González, Nov 13 '11 at 16:45

xmlHttpRequest displays eurosign as questionmark

2 Answers2

Linked