7

While reading from database the logs in my dao class shows junk characters in place of unicode characters. Sql developer shows correct values from oracle database also correct NLS language encoding is set on the database.

Below code works for standard jdbc:

connection = DriverManager.getConnection(
"jdbc:oracle:thin:@localhost:1521:xe", "adminuser", "oracle");
Statement st=connection.createStatement();
ResultSet res=st.executeQuery("SELECT menu_item_name from pending_menu_item 
where menu_item_id=6062");

    while(res.next()){
        System.out.println("itemName: "+res.getString(1));
    }

Below is the url for springboot project which shows junk characters, I uploaded to git hub.https://github.com/AyubOpen/spring-boot-jdbc/

package com.mkyong;

import com.mkyong.dao.CustomerRepository;
import com.zaxxer.hikari.HikariDataSource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.jdbc.core.JdbcTemplate;
import javax.sql.DataSource;
import java.util.List;
import static java.lang.System.exit;

@SpringBootApplication
public class SpringBootConsoleApplication implements CommandLineRunner {

@Autowired
DataSource dataSource;

@Autowired
private CustomerRepository customerRepository;

@Autowired
private JdbcTemplate jdbcTemplate;

  public static void main(String[] args) throws Exception {
      SpringApplication.run(SpringBootConsoleApplication.class, args);
}

@Override
public void run(String... args) throws Exception {

// If you want to check the HikariDataSource settings
 HikariDataSource newds = (HikariDataSource)dataSource;
  System.out.println("getMaximumPoolSize = " + ((HikariDataSource) 
  dataSource).getMaximumPoolSize());
  System.out.println("DATASOURCE = " + 
  newds.getDataSourceProperties().getProperty("hikari.useUnicode"));

  if (args.length <= 0) {
    System.err.println("[Usage] java xxx.jar {display}");
   } else {
        if (args[0].equalsIgnoreCase("display")) {
        System.out.println("Display items...");
        List<String> list = customerRepository.findAll();
        list.forEach(x -> System.out.println(x));
      }
      System.out.println("Done!");
   }
  exit(0);
 }
}

package com.mkyong.dao;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Repository;
import java.util.List;

@Repository
public class CustomerRepository {

   @Autowired
   private JdbcTemplate jdbcTemplate;

   public List<String> findAll() {

    List<String> result = jdbcTemplate.query(
        "SELECT menu_item_name from pending_menu_item where 
            menu_item_id=6062",
        (rs, rowNum) -> rs.getString("menu_item_name")
    );
  return result;
  }
}

application.properties  
----------------------
spring.main.banner-mode=off
spring.datasource.initialize=true
spring.datasource.type=com.zaxxer.hikari.HikariDataSource
spring.datasource.url=jdbc:oracle:thin:@localhost:1521/xe
spring.datasource.username=jahezdbapp
spring.datasource.password=oracle
spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver 
spring.datasource.hikari.useUnicode=true
spring.datasource.hikari.characterEncoding=utf-8 
spring.datasource.hikari.characterSetResults=utf8 

# HikariCP settings
#60 sec
spring.datasource.hikari.connection-timeout=60000
# max 5
spring.datasource.hikari.maximum-pool-size=5
Ayub
  • 599
  • 3
  • 10
  • 24
  • Possible duplicate of [HikariConfig spring boot application showing ??? characters instead of unicode](https://stackoverflow.com/questions/44985168/hikariconfig-spring-boot-application-showing-characters-instead-of-unicode) – Abhijit Sarkar Jul 09 '17 at 03:57
  • @AbhijitSarkar is there a way to delete the previous question ? I am looking at spring boot with HikariCP to work with unicode characters – Ayub Jul 09 '17 at 04:07
  • https://meta.stackexchange.com/questions/96281/how-to-close-a-question-in-stackoverflow-com – Abhijit Sarkar Jul 09 '17 at 04:08
  • @AbhijitSarkar I have delted the previous one, any clues on this question ? I am looking at example for spring boot/spring mvc + HikariCP for unicode with oracle thin driver. – Ayub Jul 09 '17 at 04:15
  • Why don't you use `oracle.jdbc.pool.OracleDataSource` when you're connecting to Oracle instead of a third-party data source? See [this](http://docs.spring.io/spring-data/jdbc/docs/current/reference/html/orcl.datasource.html) – Abhijit Sarkar Jul 09 '17 at 10:19

5 Answers5

6
  1. Add ?useUnicode=yes&characterEncoding=UTF-8 at the end of spring.datasource.url.

  2. Set spring.datasource.sqlScriptEncoding=UTF-8 in application.properties

1 itself should solve the issue, 2 may not be necessary.

Abhijit Sarkar
  • 21,927
  • 20
  • 110
  • 219
  • tried (1) spring boot fails to initialize connection spring.datasource.url=jdbc:oracle:thin:@localhost:1521/xe?useUnicode=yes&characterEncoding=UTF-8 – Ayub Jul 09 '17 at 04:39
  • Check the URL **closely**, it's not what I said – Abhijit Sarkar Jul 09 '17 at 04:40
  • 2
    I tried both & needs to be escaped as I am defining them in the .propeties file and replacing using property place holder. spring.datasource.url=jdbc:oracle:thin:@localhost:1521/xe?us‌​eUnicode=yes&cha‌​racterEncoding=UTF-8 and spring.datasource.url=jdbc:oracle:thin:@localhost:1521/xe?us‌​eUnicode=yes&cha‌​racterEncoding=UTF-8 – Ayub Jul 09 '17 at 10:01
  • 1
    connection works only if I remove the extra stuff added – Ayub Jul 09 '17 at 10:10
  • @Ayub, No to be escaped in property file, `&` is a special character only for XML. Post the stacktrace in a gist. – Abhijit Sarkar Jul 09 '17 at 10:14
  • Below is the gist url, this happened even if I remove the escape and use & as is. https://gist.github.com/anonymous/9fd54ad4db4de4b5d76b96f070b78930 – Ayub Jul 09 '17 at 10:35
  • @Ayub Did try with `OracleDataSource` like I suggested in a comment above? It's possible that HIkari doesn't understand the connection string. – Abhijit Sarkar Jul 09 '17 at 17:47
  • tried the oracle.jdbc.pool.OracleDataSource same issue below is gist of the code which produces ??? below is the gist for code https://gist.github.com/AyubOpen/9b6bb12ae3ecf6ba247399de95f42b39 – Ayub Jul 10 '17 at 10:39
  • this issue does not seem to be related to Datasource, even if I use hardcoded unicode values it still outputs ???, I have created another question with my findings https://stackoverflow.com/questions/45022387/spring-boot-commandline-runner-is-using-windows-default-character-encoding – Ayub Jul 10 '17 at 22:08
  • @Ayub Go ahead and close this question then, or we will mark the other one as a duplicate. – Abhijit Sarkar Jul 10 '17 at 22:43
2

Especially for Oracle. At least I was in the same situation.

It seems, your database uses some national (non-unicode) encoding. So JDBC cannot translate it to UNICODE.

Oracle JDBC by default supports only few character sets: US7ASCII, WE8DEC, WE8ISO8859P1, WE8MSWIN1252, and UTF8. So all strings encoded in different encodings will be displayed as questions. To add support of all other character sets add to the application classpath the file orai18n.jar.

See more here: https://docs.oracle.com/database/121/JJDBC/global.htm#JJDBC28643

xuthus
  • 144
  • 1
  • 12
  • Thank you very much for the clarification. I was struggling few days with this problem. I have an issue with questionmarks just when I was working with custom oracle types. – Milko Lorinkov Oct 07 '19 at 08:14
0

This issue does not seem to be with jdbc datasource, I will create a simple github project for springboot and followup on this issue with the below question:

spring boot commandline runner is using windows default character encoding

Ayub
  • 599
  • 3
  • 10
  • 24
0

Record: worked url for DB2, jdb

jdbc:db2://x.x.x.x:5010/xxxx:useUnicode=yes;characterEncoding=UTF-8;

todaynowork
  • 976
  • 8
  • 12
0

Had the same issue when reading unicodes. Add this line to the .bat file -Dfile.encoding=UTF8