Can't replace this unicode (�) to " " (space) on csv upload to PHP

Question

I have a Excel data converted from XLSX to CSV, then I need to upload it to my site. The data shown like this on CSV but changed after upload.

// On Excel (CSV)
Row   Description
 1    Enjoy this life without drugs
 2    Life is so short, so enjoy it

After uploading to site and inserted to MySQL it's look like this.

// On MySQL
Row   Description
 1    Enjoy?this life without?drugs
 2    Life is?so?short, so?enjoy it

// On PHP ( (echo loop).
Row   Description
 1    Enjoy�this life without�drugs
 2    Life is�so�simple, so�enjoy it

I was checked on my CSV, it just space that changed into ? and �. So, I'm trying to replace that but all failed using :

// $the_string = Line of text Description.
1. str_replace("�", " ", $the_string);
2. str_replace("&#65533", " ", $the_string);
3. str_replace("&#xfffd", " ", $the_string);
4. str_replace("?", " ", $the_string");

But, If I'm test it only on <?php str_replace("�", " ", "a�b"); ?>, It's working.

I don't know where is the mistake.

This is my source code :

public function upload()
{
   $config = array(
      "upload_path" => "./uploads/",
      "allowed_types" => "csv"
   );
   $this->load->library("upload", $config);
   $this->load->helper("file");
   $this->upload->initialize($config);
   $upload = $this->upload->data();
   $file = base_url()."uploads/{$upload['file_name']}";
   $file_handle = fopen($file, "r");
   $check_line = 0;
   while ( ! feof($file_handle))
   {
      $line_of_text = fgetcsv($file_handle, 1024);
      $check_line++;
   }
   fclose($file_handle);
   if ($check_line > 1)
   {
      $file_handle2 = fopen($file, "r");
      while ( ! feof($file_handle2))
      {
         $line_of_text = fgetcsv($file_handle, 1024);
         $description = $line_of_text[1];
         $this->model->insert_description($description);
      }
      fclose($file_handle);
   }
}

Think it's encoding problem. The same question [here](https://stackoverflow.com/questions/13298353/php-fgetcsv-charset-encoding-problems) — V. Buriy, Jan 26 '18 at 10:40

user2342558 · Accepted Answer · 2018-01-26T10:36:50.273

1

Try with preg_replace():

preg_replace('/\x{FFFD}/u', ' ', $the_string);

Try it here.

Attention: This will remove the � character from the string, but ONLY if it is the real character stored in the string.

The � character may appear in substitution of every character that isn't encoded properly accordingly with the encoding used by, in this case, PHP.

To remove all non-printable characters use this:

preg_replace('/[\x00-\x1F\x7F-\xFF]/', '', $the_string);

Try it here.

edited Jan 26 '18 at 10:36

answered Jan 26 '18 at 10:21

user2342558

5,567
5
33
54

Hi, I was try your code. The data is gone, only show 1 space on echo. I think it's same like str_replace, only working on `` check – Zinc Jan 26 '18 at 10:28
You must use the `$the_string` variable instead of `$str` of my example. I edited my answer according to your variable's name. – user2342558 Jan 26 '18 at 10:29
Sure, I was try that. – Zinc Jan 26 '18 at 10:30
Had you tried now with my last edit (with `$the_string`)? – user2342558 Jan 26 '18 at 10:30
I was tried to use `$the_string` and `$line_of_text[1]`. The data is gone, just show `" "` (1 space). I still don't know why. – Zinc Jan 26 '18 at 10:32
That is the same case like : If I'm test it only on ``, It's working and print `a b`. – Zinc Jan 26 '18 at 10:34
Success ! Thank you very much ! – Zinc Jan 26 '18 at 10:41
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/163938/discussion-between-zinc-and-user2342558). – Zinc Jan 26 '18 at 10:56

Can't replace this unicode (�) to " " (space) on csv upload to PHP

1 Answers1