0

I've been trying to migrate my MySQL to Postgres, and from what I found, the taps gem.

Unfortunately, it is failing on basically every single table with user data in it with some sort of this exception

PG::Error: ERROR:  invalid byte  sequence for encoding "UTF8": 0xed2020 (Sequel::DatabaseError)

The invalid character is different in every table, but the exception is the same.

This is not a duplicate of other PG::Error: ERROR: invalid byte sequence for encoding questions, since every single one of them mentions the use of Iconv.iconv("UTF8...", content) to force the encoding. But I can't do that since I'm using taps for the import.

Also one side note, I tried doing mysqldump --compatible=postgres, but it fails on every single query when I try to import it back into Postgres.

I am using MySQL 5.1.63 and PostgreSQL 9.1.4.

Jakub Arnold
  • 85,596
  • 89
  • 230
  • 327
  • pass encoding while running a server as - taps server mysql://root:root@localhost/tco?encoding=UTF8 root root – ajahongir Jan 04 '13 at 06:12

2 Answers2

1

You could try creating your PostgreSQL database with the SQL_ASCII encoding instead of UTF8 to see if that works around the problem.

Basically, it looks like your problem is that your MySQL database has non-UTF8 encoded text in it, and you are trying to import that into a PostgreSQL database that uses a UTF8 encoding. If you really want to keep PostgreSQL database in UTF8 encoding, you'll need to transcode whatever encoding you are using in MySQL to UTF8 before attempting to insert it.

That assumes the problem is at the database level. If the problem is at the ruby level, you could try using ruby 1.8 (if you are using ruby 1.9) and see if that makes a difference.

Jeremy Evans
  • 11,959
  • 27
  • 26
1

See this issue: https://github.com/ricardochimal/taps/issues/110

It should help to add an explicit utf-8 encoding setting when starting the taps server:

taps server mysql://user:password@host/dbname?encoding=utf8 httpuser httpass
ohcibi
  • 2,518
  • 3
  • 24
  • 47