0

What is the recommended way to import a data dump into a Free Tier (Sandbox) cluster on YugabyteDB Managed ? I have a relative small DB dump using ysql_dump from my local YB docker, then I use ysqlsh to import the dump to YB managed free cluster but get the following error:

ssl syscall error eof detected connection to server was lost

But if I import the dump to YB in docker locally, it is fine. Is it because of the import is taking too long?

[Disclaimer]: This question was first asked on the YugabyteDB Community Slack channel.

MarkoR
  • 115
  • 8

1 Answers1

0

If this is happening repeatedly the connection is more than likely dropping due to OOMs on the Sandbox cluster. Remember that these Sandbox instances are only meant for prototyping and getting familiar with YugabyteDB. They are not meant for testing and Proof of Concepts. Since they have less than 4GB of RAM any memory intensive operation may cause these types of issues you are seeing. This is usually seen in scenarios with too many concurrent connections, large amount of tables and/or indexes, as well as large tables with millions of rows. These limitations are called out in our docs.

Another potential issue could be a table with a large amount of columns. If you are using the COPY command (or using a tool that uses the COPY command behind the covers) there is a chance that this is the cause of the issue. The COPY command inserts data in a single transaction up to the setting of rows_per_transaction, which is 20k by default in newer versions. The combination of a large amount of columns and the max amount of rows being inserted in a single transaction can cause too much load on this memory constrained Sandbox cluster.

There are a couple workarounds in these cases:

  • Lower the value of rows_per_transaction to a threshold that does not cause OOMs. This will depend on the amount of columns on the table, their types, and the length of those values. For example columns with blob types or lengthy strings will be more likely to cause issues. Here is an example of changing this variable.
  • In cases where the above change is not applicable, you can open the import file and manually split the COPY command into multiple COPY commands.
  • Request a Free Trial on YugabyteDB Managed which includes month long testing on a 3 node cluster.
MarkoR
  • 115
  • 8