0

I am trying to merge two .Xdf file, below is the code for the same:

rxMergeXdf(inFile1 = "/poc/revor/data/se_lif_1.xdf",
inFile2 = "/poc/revor/data/ext_merchant_account_filtered.xdf", 
outFile = "/poc/revor/data/se_lif_2.xdf", matchVars = c("SE_NO"), 
type = "inner", missingsLow = TRUE,decreasing = FALSE,verbose = 1,
overwrite = TRUE,rowsPerOutputBlock = -1)

But getting this Error while execution :

Memory buffer limit: 723679 Mb.
Merge Type: 1, Inner
Time to merge data file: 0.025 seconds
Merged data file: /poc/revor/data/se_lif_2.xdf
Error in rxCall("RxMergeXdf", params) :

Output .xdf file is created with size 260 bytes but not able to open it. Its showing below message when try to get details of this .Xdf using rxGetInfoXdf:

Error in rxCall("RxReadXdf", params) :
  This is an old version of a DataFile, and cannot be read.

Below is the file structure for the two .Xdf file:

1) se_lif_1.xdf

File name: /poc/revor/data/se_lif_1.xdf
Number of observations: 832787765
Number of variables: 2
Number of blocks: 4
Compression type: zlib
Variable information:
Var 1: SE_NO, Type: numeric, Low/High: (1030026553.0000, 9992702050.0000)
Var 2: TOT_CV_LIF, Type: numeric, Low/High: (-2348.7600, 40587.3900)
Data (10 rows starting with row 1):
        SE_NO TOT_CV_LIF
1  1030026553     314.45
2  1030049191      60.00
3  1030227870    2643.71
4  1040124257     389.00
5  1040608119      85.59

2) ext_merchant_account_filtered.xdf:

File name: /poc/revor/data/ext_merchant_account_filtered.xdf
Number of observations: 10000
Number of variables: 2
Number of blocks: 4
Compression type: zlib
Variable information:
Var 1: SE_NO, Type: numeric, Low/High: (3047490719.0000, 9380463753.0000)
Var 2: SEIMS_INDUS_DS_CD, Type: character
Data (10 rows starting with row 1):
        SE_NO SEIMS_INDUS_DS_CD
1  5510918899               499
2  3132775164               915
3  4107745093               930
4  3125354860               248
5  5510918923               499

Could any one help me out with this issue. I looked at the code but every thing looks fine to me.

Thanks for your help in Advance...:)

Cheers,

Amit

Amit Mishra
  • 33
  • 1
  • 10
  • It's possible your xdf files have become corrupted. See if reading them into memory as data frames works. You can use `rxReadXdf` with the `numRows` argument to limit memory usage if it's a problem. – Hong Ooi Jun 23 '15 at 01:56

1 Answers1

0

I faced the same problem when I was merging 2 data frames. The problem was occurring when there are no matching values of the merging variable of the left dataframe in the right one ie., the output data frame is empty. The low/high value ranges might overlap, but there might be no matches in 'matchVars'.

Sai Bharadwaja
  • 1
  • 1
  • 3